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Abstract 

In the Maslov idempotent probability calculus, expectations of random variables 
are denned so as to be linear with respect to max-plus addition and scalar multipli- 
cation. This paper considers control problems in which the objective is to minimize 
the max-plus expectation of some max-plus additive running cost. Such problems 
arise naturally as limits of some types of risk sensitive stochastic control problems. 
The value function is a viscosity solution to a quasivariational inequality (QVI) of 
dynamic programming. Equivalence of this QVI to a nonlinear parabolic PDE with 
discontinuous Hamiltonian is used to prove a comparison theorem for viscosity sub- 
and super-solutions. An example from math finance is given, and an application in 
nonlinear .ff-infinity control is sketched. 

Key words: Max-plus control, max-plus additive cost, risk-sensitive stochastic 
control, nonlinear parabolic PDEs, viscosity solutions. 
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1 Introduction 

A wide variety of asymptotic problems, including large deviations for stochastic processes, 
can be considered in the framework of the Maslov idempotent probability calculus. In this 
theory, probabilities are assigned which are additive with respect to "max-plus" addition 
and expectations are denned so as to be linear with respect to max-plus addition and 
scalar multiplication. For this reason we use the term "max-plus probability" instead 
of "idempotent probability." There is extensive literature on max-plus probability and 
max-plus stochastic processes. See [2], [3], [6], [TO], p2] and references cited there. The 
max-plus framework is also important for certain problems in discrete mathematics and 
in computer science applications. See PQ, [3] for example. 
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To provide background for the results of this paper, let us begin by mentioning two re- 
sults from the Freidlin-Wentzell theory of large deviations for small random perturbations 
of dynamical systems [2]. Consider a finite time interval [t, T] and X(s) which satisfies 
the stochastic differential equation (SDE) 

dX(s) = f(X(s))ds + 6- 1/2 a(X(s))dW(s), t < s < T, 
X(t) = x e R n . 

Here 9 > is a "large" parameter and W(s) is a d- dimensional Brownian motion. Some- 
times we write X(s) = Xg(s) to emphasize dependence on 9. Asymptotic large deviations 
results as 9 — ► oo are typically described through a deterministic optimization problem. In 
the limit problem, Xq(s) in (II .ip is replaced by x(s), which satisfies an ordinary differential 
equation (ODE) 

dx 

— (a) = f(x(s)) + a(x(s))v(s), t<s<T, 
x{t) = X. 

The unknown function v(-) is a "control," with v(-) G L 2 ([t,T];M. d ). In the language of 
nonlinear iZ-infinity control theory, v(s) is called a "disturbance" [15]. The disturbance 
control is chosen to maximize an expression of the form $(x(-)) — o ll v C*) II2 with || • H2 the 
L 2 -norm. The maximum is the max-plus expectation E + [Q(x(-))} as defined in Section 2. 

We may consider, in particular, the following two kinds of choices for $. 
Case 1 

$ 1 (a:(.)) = y l{x{s))ds + G{x{T)). 

The term G(x(T)) is called a terminal cost. When G = 0, we say that $1 is a "max-plus 
multiplicative" running cost. Under suitable assumptions on /, o and I, the Freidlin- 
Wentzell theory implies that 

lim 9- 1 log£[exp{0$i(X fl (.))}] = £ + [$!(x(-))]. (1.3) 



Case 2 



l(x(s))ds = max l(x(s)). 
[t,T] ' 



This is the case of "max-plus additive" running cost. In Case 2, the large deviations result 
(see [1 lj ) is 



lim 9~ l \ogE 



exp{9l(X e (s))}ds 



;i.4) 



In this paper, we are concerned with problems in which the time evolution of the 
state x(s) depends on a control u(s). Thus, equation (jl.2p above will be replaced by the 
ODE (12. 2p . The case of max-plus multiplicative running costs has already been studied 
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using methods of risk sensitive stochastic control theory and differential games. See [T3l 
Chap. 6], [TU]. We consider the problem of choosing the control u(s) to minimize 



E~ 



l(x(s),u(s))ds 

[t,T] 



E" 



ess. sup l(x(s),u(s)) 

t<s<T 



;i.5) 



We call this a "max-plus stochastic control problem with max-plus additive running cost." 

The control u(s) at time s is to be chosen using information about disturbances v(r) 
for times r < s. A precise definition of the class T(t,T) of admissible strategies is given in 
Section 3. T(t, T) consists of Elliott-Kalton strategies with the additional properties (SI), 
(S2). Another possible class consists of those Elliott-Kalton strategies which are "strictly 
progressive," as defined in [131 Section 11.9] and [16]. See also Section 4.3. 

The max-plus stochastic control problem is studied using the method of dynamic pro- 
gramming. The associated value function V(t, x) is defined by (13.11) . It satisfies a dynamic 
programming principle (Theorem [33]), which is proved by standard arguments using prop- 
erties of max-plus conditional expectations. The dynamic programming equation for V 
takes the form of a quasivariational inequality (QVI) (12.111) . It is shown later (Theorem 
14.91) that V is the unique bounded, Lipschitz continuous viscosity solution to (12. lip with 
boundary data (12.121) at time T. 

In Section 4, the QVI (12. lip is shown to be equivalent to the nonlinear parabolic PDE 
(14.11) . which involves a discontinuous Hamiltonian. In the special case when there are no 
disturbances in the model (a = 0), the PDE (14. ip is of a kind considered by Barron-Ishii 
[5]. The treatment of viscosity subsolutions and supersolutions in Section 4, and the proof 
of the comparison Theorem 15.11 make use of similar ideas in [5] . 

Section 4.2 considers a nonlinear two time parameter semigroup associated with the 
max-plus control problem. This semigroup is expressed in terms of a family of operators 
F t>r , which are related to max-plus linear operators. The semigroup property is a con- 
sequence of the dynamic programming principle. The operator ^ + Ti in (14. ip has an 
interpretation as the generator of this semigroup. See Theorem 14.71 

In the max-plus control problem, there are actually two controls. One is the control 
u(s), chosen to minimize (11.51) . The other (maximizing) control is v (s), which enters 
through the max-plus expectation E + . Although our problem has a differential game 
interpretation, we make no use of results about differential games. See remarks in Section 
4.3. 

In Section 6, we consider a stochastic control version of the model, in which the ODE 
(12.21) for the state dynamics is replaced by the SDE (16. ip and the goal is to minimize the 
expectation of the risk-sensitive criterion in (16. 2p . Let tyo(t,x) be the value function for 
this problem. Theorem 16.31 states that 

lim 9~ l log y (t, x) = V(t, x). (1.6) 

The corresponding large deviations result is (II .4p . Theorem 16.31 is proved using a version 
of the Barles-Perthame method for viscosity solutions. 

As an example to illustrate the limit in (II .6p . we consider in Section 7 the classical 
Merton optimal investment-consumption problem in mathematical finance. For the Mer- 
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ton problem, ^e(t, x), V(t,x) and the corresponding optimal controls can be found by 
explicit calculations. 

In Section 8 we consider some infinite time horizon problems. The interest is in inequal- 
ities of the form (18.1 ft which hold on every finite time interval [0, T\. Such inequalities have 
an interpretation in nonlinear if -infinity control theory, with max-plus additive running 
cost. The discussion follows mainly jTH Section 8]. 



2 Max-plus stochastic control 

2.1 Preliminaries on max-plus probability 

We start with reviewing some notions and facts from max-plus probability theory which 
will be needed later. The readers should refer to [2], [3], [6], [TT], [15], [TF] for more details. 

Let us consider extended reals Mr = EU {— oo}. For a, b E M.~ , we define new addition 
and multiplication by 

a © b = max{a, b}, a ®b = a + b. 

Mr with these new operations is called max-plus algebra and it satisfies all the axioms of 
rings except for the existence of additive inverse. In addition, the idempotency a © a = a 
holds. 

To introduce the notions of max-plus probability, we focus on a particular case dis- 
cussed in the present paper. Let t G [0,T] and Q = Q t ,T = L 2 ([t,T];M d ) be a sample 
space. We use the notation L 2 [t,T] for L 2 ([t,T];M d ) if the dimension is clear from the 
context. On this sample space, we consider max-plus probability density Q : Q — > M~ 

Q(v) = ~ £ \v(s)\ 2 ds, ven. 

Then, the max-plus probability P + for A C Q is defined by 

P + {A) = supQ(v) = sup {-\ f \v(s)\ 2 ds) . 

The supremum on empty set is understood to be — oo. We call (fl, P + ) (or (Q,Q)) a 
max-plus probability space. 

Let Z : Q — ► M~ be a random variable. Here we do not require any measurability 
conditions. The max-plus expectation of Z is 

E+[Z] = sup{Z(v) ® Q(v)} = sup \z{v) - \ I \v{s)\ 2 ds\ . 
ven veL 2 [t,T] I ^ Jt J 

Z is max-plus integrable if E + [Z] < oo. It is easily seen that the max-plus expectation 
is linear under max-plus algebra: for max-plus integrable random variables Z, Y and 

a e Mr, 

E + [Z ®Y]= E + [Z] © E+[Y), E + [a ®Z] = a® E + [Z}. 
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If Z < Y, then 

E + [Z] < E + [Y]. 

To consider max-plus conditional expectations under this special probability space, let 
< t < r < T and denote Q\, f2 2 by 

ill = tt t:T = L 2 [t,r], n 2 = = L 2 [r,T}. 

On the product space Qi x Q 2 , we define max-plus probability density Qi <S> Q2 by 

(Qi ®Q2)(vi,v 2 ) = Qi(ui) ®Q2(v 2 ), (v u v 2 ) eflix Q 2 , 

where 

Qi{v x ) = ~ f \v x {s)\ 2 dsi Q 2 (v 2 ) = ~J \v 2 (s)\ 2 ds. 
Note that if we set V\ = u|[t, r ], v 2 = f |[ r ,T] for 

= g 1 (vi)®Q 2 (v 2 ). 

Thus, Q) can be identified with (f^ x fi 2 , Qi (8> Q 2 ). 

Let Z(t> ) = Z(v\, v 2 ) be a random variable on Q. For ^ 6 fij, the max-plus conditional 
expectation of Z under V\ is given by 

E+[Z\vi] = sup [Z( Vl ,v 2 ) ® Q 2 (t> 2 )]. 

If Z has the form Z(vi,v 2 ) = Z~i{y\)@Z 2 {yi, v 2 ) for some Zi : fii — > ¥L~ and Z 2 : Q — > M.~, 
it is seen that 

E + [Z] = E + [Z X © E + [Z 2 (vi, -)M]. (2.1) 



2.2 Problem formulation and verification theorem 

We consider max-plus control problems on a finite time interval [t,T]. The final time T 
is fixed throughout the paper, and the initial time satisfies < t < T. Let us consider 
the system governed by 

{dx 
— (s) = f(x(s),u(s)) + a(x(s),u(s))v(s), t<s<T, 
x(t) = xeR n , 

where U C M m , / : M n x U -> M n , a : M n x [/ -> M(n, d), M(n, d) is the set ofnxd 
matrices. a?(s) is the state of the system, u G L°°([t, T]; Z7) is a control and t> G L 2 [i, T] 
is a disturbance (or uncertainty) in the system. Equation (I2.2p is an ordinary differential 
equation with two parameters. However, in terms of max-plus diffusion processes, we 
can regard (12. 2p as a controlled max-plus stochastic differential equation under (ft, Q) (cf. 

CD)- 

When we choose a control at a certain time, it is natural to require that the decision 
has to be made by past information of the disturbance. It can be realized by the notion 
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of Elliott-Kalton strategy in the theory of differential games (cf. [8]). Let a : L 2 [t,T] — ► 
L°°([t, T);U). a is an Elliott-Kalton strategy from L 2 [t, T] into L°°([t, T); U) if a satisfies 
the following: Let v , v G L 2 [t,T] and t < s < T. 

If v = v a.e. on [t, s], then a[w] = a[v] a.e.on [t,s]. (2.3) 

We denote by T EK (t, T) the set of Elliott-Kalton strategies from L 2 [t, T] into L°°([t, T);U). 
If we choose a G Fek^jT), (I2.2p becomes 

— (s) = /(z(s),a[<u](s)) +<7(x(s),a[u](s))T;(s), t<s<T, 

x(t) = x e« n 

for each disturbance v G L 2 [t,T]. 

The goal in max-plus stochastic control is to minimize the max-plus expectation of 
some criterion J on a suitable subclass of Elliott-Kalton strategies. For the controlled 
system (12.41) . there can be three natural criteria: 

(i) Terminal cost: For $ : W 1 -> R, 

J = ${x{T)). 

(ii) Max-plus multiplicative running cost: For / : IR n x U — > R, 

J= / l(x{s),a[v](s))ds. 



(iii) Max-plus additive running cost: For I : R n x {/ -* K, 

J— I l(x(s), a[v](s))ds = ess. sup l(x(s), a[v](s)). 

J[t,T] se[t,T] 

(i) and (ii) are considered in [131 Section 11.7] and [IB] . In the present paper, we shall 
discuss the criterion of type (iii). 

Unless otherwise stated, we assume that the following conditions hold: 
(Al) U C R m is compact. 

(A2) / and a are of C 1 on IR n x (/. /, a and their derivatives f x , a x in x are bounded on 
W 1 x [/. 

(A3) / is of C 1 on R n x [/. /, l x and i u are bounded on W 1 x [/. 

Under (A2), there exists a unique solution of (12.21) for any control u G L°°([t,T]; U) and 
disturbance v G L 2 [t,T]. 
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We formulate our max-plus control problem more specifically. Let a subclass T(t,T) C 
^EK{t,T) be given. For a E T(t,T), consider the max-plus expectation of the max-plus 
additive running cost criterion: 



J(t, x; a) = E. 



tx 



sup 



l(x(s), a[v](s))ds 



[t,T] 



l(x(s), a[v }(s))ds — 



1 



\v(s)\ 2 ds\, 



(2.5) 



v£L 2 [t,T] U[t,T] 

where x(s) is the solution of (12.41) . We indicate the dependence on the initial condition 
of the system by the subscript tx of Ef x . Our concern in max-plus stochastic control is 
to minimize J(t, x;a) on T(t,T). Thus, the value function associated with strategy class 
T(t,T) is defined by 



V(t, x) = inf J(t, x; a) = inf E } 



a£T(t,T) 



tx 



<*er(t,T) U[t,T\ 



l(x(s), at[v](s))ds 



(2.6) 



For given class T(t, T) C V ek{^i T), it is a fundamental problem to characterize V(t, x) 
as a (unique) solution of the associated dynamic programming equation (DPE). In order 
to guess the DPE for V(t,x), consider J(t, x;a) with constant strategy a[u](s) = u for 
u EU, i.e, let us define V u (t,x) by 



V u (t, x) = E. 



tx 



l(x(s), u)ds 



[t,T] 



(2.7) 



In [TT], it is proved that under (Al)-(A3), V u (t, x) is the unique bounded Lipschitz 
continuous viscosity solution of 

dV u 



max 



dt 



+ H u (x } VV u {t, x)), l(x, u) - V u (t, x)}=0, (t, x) E (0, T) x 



{21 



V u (T,x) = l(x,u), x E 
where for x, p E M. n and u E U, 



H u (x,p) = max < (f(x, u) + a(x, u)v) ■ p \v\ 

v£R d | ' 2 



(2.9) 



-a(x, u)p ■ p + f(x, u) ■ p, a(x, u) = a(x, u)a(x, u) J 



Note that a(x,u) T p attains the maximum in H u (x,p), i.e., 

a(x, u) T p E arg max j (f(x, u) + a(x, u)v) ■ p — - \v\ 2 



(2.10) 



We expect that a value function V(t,x) for some strategy class may satisfy the following 
quasivariational inequality (QVI): 

[dV 1 

minmax — + H u (x, VV{t,x)),l(x,u) - V(t,x) > = 0, (t, x) E (0, T) x R n , (2.11) 
ugu at 



V(T, x) = min l(x, u), xE 



(2.12) 
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We will see that (12.111) with (12.121) is the correct DPE for our max-plus control problem 
in (12. 6p if T(t,T) is properly chosen. In Section 4, we will show that (12. lip is equivalent 
to a nonlinear parabolic PDE with discontinuous Hamiltonian. When T(t, T) is chosen to 
satisfy (SI), (S2) in Section 3, then the value function is the unique bounded Lipschitz 
viscosity solution of (12.111) . (I2.12p . See Theorem 14.91 

We conclude this section by considering strategies determined by Markov control poli- 
cies u. Let u(s, y) be a Lipschitz continuous function of (s, y) £ [t, T] x M. n into U. The 
corresponding strategy a- is defined by 



and a-[u](s) = u(s, x(s)). u is called a Markov control policy. 

Theorem 2.1. Let W(t,x) be a ^-solution of fl2TTPp and f[2TT2D such that VW(t,x) 
satisfies a uniform Lipschitz condition on x. Then: 

(a) W(t,x) < J(t,x; a-) for every Lipschitz Markov control policy u; 

(b) If there exists a Lipschitz Markov control policy u* such that for any (s, y) £ [t, T] x M n 



u*(s, y) £ argminmax <^ -— (s, y) + H u (y, VW(s, y)), l(y, u) - W(s, y) \ , (2.14) 




-(s) = f(x(s),u(s, x(s))) + a(x(s),u(s, x(s)))v(s), t < s < T, 
s 

x(t) = X 



(2.13) 




then W(t,x) = J(t,x;a-*). 



Proof. We shall first show (a). By the fundamental theorem of calculus, 




dW 



W(t,x) + 



dr 



(r,x(r)) + (f(x(r),a-[v](r)) + a(x(r),a-[v](r))v(r)) ■ VW(r, x(r)) 




where x{r) is the solution of (12.131) . 

Let us consider the following closed system on [t,T]: 




~( s ) = f( x ( s )il±.( s i x ( s ))) + ^(x(s),u(s, x(s)))a(x(s),u(s, i(s))) T Viy(s, x(s)) 
s 

x(t) = x. 



(2.16) 



Define v(-) by 



v(r) = o~(x(r),u(r, x(r))) T VW(r, x(r)), t <r <T. 
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From (jg7T5D with v(-) = v(-), 

1 f s 

W(s, x(s)) - - \v(r)\ 2 dr 

2 Jt 

= W(t,x) + J l—(r,x(r)) + (f(x(r),a^[v](r))+a(x(r),a^[v](r))v(r)) ■ VW(r,x(rj) 

- l -\v{r)\^dr. (2.17) 

By noting ( 12 . 1 Of) . we have 

1 f s 

W(s,x(s)) - - / \v(r)\ 2 dr 

2 Jt 

= W(t,x)+J |^(r,x(r)) + i7 a - [{i](r) (£(r),V^(r,x(r)))|rfr. (2.18) 

We shall consider two cases: Suppose that 

— (r, x{r)) + H a ~^ r \x(r), VW(r, x(r))) > 0, t<r <T. 
Then, by f l2~T8l) with s — T, 

f-T 

2 „ 

From fl2~T2|) . 

iy(T,x(T)) = minZ(x(T),u) < l(x(T),a^[v](T)) 



W{T,x{T))-\ I \v(r)\ 2 dr > W{t,x). 



< sup l(x(s),aP[v](s)) = / l(x(s),aP[v]{s))ds. 

t<s<T J [t.T] 

Here note that ess. sup on [t,T] coincides with sup on [t,T] because l(x(s), a-[v](s)) is 
continuous on [t,T]. Thus we have 



f® i r T 

W(t,x)< l{x(s),aP{v]{s))ds-- \v(s)\ 2 ds < J(t,x;aF). 

J[t,T] 2 J t 



We next consider the case where there exists ro & (t, T) such that 
(W ir ,x(r )) + H a ^ ro \x(r ),VW(r ,x(r ))) < 0. 



dr 

Define r by 



r = inf |rG [t,T] ; ^(r, x(r)) + F a ^ 1(r) (x(r), W(r, x(r))) <0 



We can show that the following claims hold: 

^(r, x(r)) + H a ^ r \x(r), VW(r, x(r))) > 0, t < r < r, (2.19) 
or 

l(x(T),a[v}(r)) > W(t,x(t)). (2.20) 

(I2.19p is obvious from the definition of r. For the proof of (12.201) . since W(t,x) is a 
solution of fl2TTTD . 

(dW 1 

max < -Q^-ir, x) + H u (x, VW(r, x)), l(x, u) - W(r, x) > 0, Vm G U. 



In particular. 

dW 



max 



(r,x(r)) + /7 a " [{i]W (x(r), VW(r,x(r))),l(x(r),o^[v](r)) - W(r,x(r)) \ > 



dr 

t<r <T. (2.21) 

Take a sequence {r n } such that r n { r {n f oo) and 
dW 



dr 

Thus, from (1221)1 . 



(r n ,x(r„)) + ^° Ii[el(r " ) (£(^), V^(r„,x(r n ))) < 0, Vn. 



Z(x(r n ),a^](r n )) > W(r ft , x(r„)), Vn. 
Since s i— > /(£(s), a-[-0](s)) = l(x(s),u(s, x(s))) is continuous, we have by taking n — > oo 

Z(x(t),o^](t)) > W(t,z(t)). 
In fl27TS|> . if we take s = r, 

W{t,x{t))-- [ \v{r)\ 2 dr 
2 ./+ 



By (1235)1 and (1220)1 . 



W(t,x) + / (^(^(rjj + ^WfxW.V^itr)))}*. 



/(x(r),a^](r)) - - / \v(r)\ 2 dr > W{t, x). 



2J t 

If we make the dependence on a-[v] and v clear, this inequality can be written by 

l(x a -W>%T),cP[v)(T)) - \ f \v(r)\ 2 dr > W(t,x) (2.22) 

2 Jt 

where x a ~^' v is the solution of (12.41) corresponding to ai-[t>], v. 
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Define v : [t, T] — > R n by 



us 



f)(s), t < S < T, 
0, t < s<T. 



Then, from the definition of a-, we can see that 

x a - [ ^{r) = x c 
Therefore, from f[2~22D . 

W{t,x) < l(x a ^(r),a^[v](r)) - \ f \v{r)\ 2 dr 

2 Jt 

</ Z(x Q *(s),Q;^](s))tis- - / |5(r)| 2 dr < J(t,x;a^). 

J[t,T] 2 y t 

We shall now prove (b). By (127131) . we have for (s, y) £ (t,T) x R n 

max -(., y) + H* (s ^(y, y)), l(y, u*(s, y)) - W(s, y)j = Q. 

This implies that 

— (s, y) + H- l'>v\y, VW(s, y)) < and l(y, u*(s, y)) - W(s, y) < 0. (2.23) 

For any v £ L 2 [t,T], consider fl2TT3D with u(s,y) = u*(s,y). By fl2TToD with u(s,y) = 
u*{s,y), 

1 f s 

W(s,x(s)) - - / |v(r)| 2 dr 

2 it 

= W(t,x) + J | — (r,x(r)) + (f(x(r),aP [v](r)) + a(x(r), a* [v](r))v(r)) ■ VW(r,x(r)) 



2 

<W(t,x) + / \ ^(r,x(r)) + H a - [v ^ r \x{r),VW{r,x{r-))) \ dr. 



Thus, from fl2~23|) . 



1 /" s 

/(x(s),c^>](s)) - - / |w(r)| 2 dr < t<s<T. 



Hence we have 



i r T 

l(x(s),c^[v](s))ds- - / \v(r)\ 2 dr < W(t,x). 

[t,T] 2 J t 

Since t> £ L 2 [t,T] is taken arbitrarily, 

J(t,x;c^*) = sup If l(x{s),a^[v){s))ds-l f \v(s)\ 2 ds\ < W(t, x). □ 

veL 2 [t,T] U[t,T] 2 J t J 
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Remark 2.2. (i) Theorem 12.11 holds under weaker assumptions than (Al)-(A3). An 
inspection of the proof shows that Theorem 12.11 is true if we omit (Al) and also the 
assumption that / is bounded in (A2). Instead of (A3), I can be any continuous function. 

(ii) Theorem 12.11 (a) can be extended to a general class including Markov control policies. 
If we consider any strategy a satisfying (SI), (S2) (Section 3) and a[v](T— ) = a[f](T), 
we can have W(t,x) < J(t,x;a) under more regularity assumptions on W(t,x). This 
can be shown with modifications of the proof of Theorem 12.11 by using Lemma 14.81 for a 
particular Elliott-Kalton strategy f3 : L°°([t,T]; U) -> L 2 [t,T}; 

j3[u](s) = a(x(s),u(s)) T VW(s,x(s)), t<s<T, u G L°°([t,T];U), 

where x(s) is the solution of (12.21) with v(s) = f3[u](s). If we use ct[v e ] and v e (see Lemma 
14.81) instead of u and v, respectively, the arguments in the proof are still valid by working 
with additional discussions on approximations. Such idea is also used in the proof of 
g^O]). 

3 Value function and dynamic programming princi- 
ple 

To characterize a value function as a solution of the DPE, we usually need (i) the dy- 
namic programming principle (DPP) for the value function, (ii) the correct form of the 
infinitesimal generator of the semigroup associated with the DPP. Under (i) and (ii), one 
could show that the value function is a solution of the evolution equation in some sense 
(cf. [TBI Chapter 2]). In this section, we shall introduce a strategy class which will be 
related to (12XL]) and show the DPP. 

As a strategy class for our max-plus control problem, we consider T(t,T) C T EK (t,T) 
defined by the set of a : L 2 [t,T] — > L°°([t, T]; U) satisfying the following conditions: 

(51) For any v G L 2 [t, T], s i— > a[v](s) is right-continuous with left limits on [t,T\. 

(52) Let v, v G L 2 [t,T] and t < s <T. If v — v a.e.on [t, s], then a[v] = a[v] on [t, s]. 

Note that we require a[t>](r) = a[?)](r) for all r G [t, s] in (S2) (compare with (12.31) for 
Elliott-Kalton strategy). We always take T{t,T) satisfying (SI) and (S2) for our strategy 
class in the rest of the arguments. 

We point out some properties on instantaneous time delay of T{t,T). The proofs are 
immediate from the definition. 

Lemma 3.1. For a G T(t,T), (i) and (ii) hold: 

(i) a[w](t) does not depend on v G L 2 [t,T}. 

(ii) Letv, v G L 2 [t,T] and t < s <T. If v = v a.e. on [t,s), then a[v](r) = a[v](r) for all 
r G [t, s] . 
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Remark 3.2. If u is any Lipschitz Markov control policy (Section 2) and we consider 
r(t, T) satisfying (SI) and (S2), then a- G T{t,T). On the other hand, if a[f](s) = 0[v(s)] 
for some nonconstant Borel measurable function <ft: M. d — > U, then a G r##(£, T) but 
a ^ T(t, T). For any a G T(t, T) and t < s < T, a[v] has the left hand limit a[v](s— ) which 
depends only on v(r) for r < s. Moreover, a[f](s— ) = a[t>](s) except for countably many 
s. This expresses (in a mathematically imprecise way) the intuitive idea that condition 
(SI) allows ot[v](s) to depend on past values v(r) for r < s, but not on the current value 
v(s). 

For t <T, let V(t,x) be the value function associated with T(t,T), i.e., 



V(t,x)= inf 
aer(t,T) 



inf sup 



[t,T] 



l(x(s), a[v](s))ds 



aer(t,T) „ G L 2 [t,T] U[t,T] 



l(x(s), a[v](s))ds — - / \v(s)\ 2 ds 



(3.1) 



where x(s) is the solution of ( 12.41) . When t = T, we let T(T,T) = U and define V(T,x) 
by 

V(T, x) = min Z(x, u). 

As the following theorem shows, we have the DPP for V(t,x). 
Theorem 3.3. For any (t,x) G [0,T] x IT and t < r <T, 



V(t,x)= inf E+ 

aeV(t,r) 



t,r] 



l(x(s),a[v](s))ds © V(r,x(r)) 



(3.2) 



Proof. In the case where t = r = T or t = r < T, (13.21) is immediate. 

We next consider the case where t < r < T. Let W(t, x) be the right-hand side (RHS) 
of (13. 2p . As in Section 2, we identify v with (t>i,t>2), where Vi = v\[ t>r ] and v 2 = v\\ r> T\. For 
any e > 0, we take a 1 G T(t,r) such that 



W(t,x) + e>E+ 



l(x(s),a 1 [v 1 ](s))ds © V(r,x(r)) 



[t,r] 



(3.3) 



where x(s) (t < s < r) is the solution of (12.21) on [£, r] for u = c^ffi] and v — V\ G £ 2 [t, r] 
For each £ G M n , choose a| G T(r, T) such that 



r,T] 



Z(x 2 (s), a?[f 2 ](s))cis 



where x 2 (s) (r < s < T) is the solution of (12.21) on [r, T] for u = a^fi^] and t> 
L 2 [r, T] with initial condition x 2 (r) = £. 

For w G L%T], define a[v] G L°°([t,T]; U) by 



(3.4) 
^2 G 



a[v](s) 



a 1 [t>i](s), t < s < r, 
« 2 (r) N( S ), r< S <T. 



(3.5) 
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x{r) is the solution of (12.21) given by u = a 1 ^] and v — V\. Note that a x ( r )[v2\{r) does not 
depend on t> 2 because of Lemma [3.11 (i). Then, it is not difficult to check that a satisfies 
(SI) and (S2), that is, a G T(t, T). 

By using (O), 



E, 



tx 



l(x(s), a[v](s))ds 



U[t,T] 



E, 



tx 



U[t,r] 



l(x(s),a 1 [v 1 }(s))ds © 



U[r,T] 



l(x(s), a[v](s))ds 



[*,r] 



l(x(s),a 1 [v 1 ](s))ds®E+ 



rx(r) 



r,T] 



l(x 2 (s),a 2 x(r) [v 2 }(s))d.< 



From (13.41) . the last term can be estimated by 



E 



tx 



[t,r] 



l(x(s),a 1 [v 1 }(s))ds®E+ 



rx(r) 



[r,T] 



l(x 2 (s),a 2 x{r) [v 2 ](s))di 



< E. 



tx 



/(x(s),a 1 [t; 1 ](s))rfs © (V(r, x(r)) + e) 



[t,r] 



< E. 



tx 



l(x(s),a 1 [v 1 ](s))ds © V(r, x(r)) 



U[t,r) 

Thus we have from (13.31) 



+ e. 



E 



tx: 



l(x(s), a[v](s))ds 



U[t,T] 

Since a G T{t,T), we obtain 
Sending e to 0, we have 



< W(t,x) + 2e. 



V(t,x) < W(t,x) + 2e. 



V{t,x) < W{t,x). 



To prove W(t,x) < V(t,x), take any a G T(t,T). We define a± : L 2 [t,r] — > 
L°°([t,r];U) by 

«i[fi](s) = a[t>i • f^s), t < s < r, v 1 E L 2 [t,r], (3.6) 

where fj? G L 2 (r, T] is a (dummy) disturbance on (r, T] and v\ ■ v\ is the concatenation of 
v i and f S: 



i>i(s), t < s < r, 
^(s), r < s < T. 



Note that «i does not depend on the choice of by (S2). Since a G T(t,T), a x G T(t, r). 
For a given fx G L 2 [t,r], we define a^ 1 ■ L 2 [r,T] —>■ L°°([r, T]; [/) as follows: 

<N(«) = afa ■ u 2 ](s), r < s < T, t; 2 G L 2 [r,T]. (3.7) 
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We can see a^ 1 G T(r, T) because a G T(t,T). Thus, by the definition of W(t,x) and 
V(r, x(r)), 



W(t,x)<^+ 

<e; 

By using (IQl 



[*,r] 



l(x(s),ai[v 1 )(s))ds © F(r,x(r)) 



[t,r 



Z(z(s), ai[vi] (s))ds©£+ (r) 



r,T] 



Z(x2(s),a2 1 [v 2 ](s))ds 



(3. 



RHS of (JSU) = 



[*,r] 



/(x(s), a^ 1 [u 2 ](s))ds 



r,T] 



«1 



[t,T] 



l(x(s), a[v](s))ds 



Thus we have 



[t,T] 



l(x(s), a[v](s))ds 



W(t,x)<E+ 

Since a G r(t, T) is taken arbitrarily, 

W(t,x) < V(t,x). 

Finally, let t < r = T. For any a G T(t,T), 



V(T,x(T)) < l(x(T),oc[v](T—)) < sup l(x(s), a[v](s)) = / l{x{s),a[v]{s))ds. 

t<s<T J[t t T] 

In this case, (13.21) is immediate from (12.61) . □ 

By using (A1)-(A3) and the DPP for the value function, we can obtain the following 
regularity result on the value function. 

Proposition 3.4. V(t,x) is bounded Lipschitz continuous on [0, T] x M n . 

Proof. It is obvious that V(t, x) is bounded. Since /, a, I are time-independent, if t < T 
then V(t, x) can be rewritten as follows: 



V{t,x)= inf E+ 



l(x(s), a[v](s))ds 



[0,T-t] 



inf sup < / l(x(s), a[v](s))ds / \v(s)\ 2 ds 

aeT{0,T-t) veL 2[ 0tT _ t] U[o,T-t] ' 2 Jo 



T-t 



(3.9) 



where x(s) is the solution of 

dx 



ds 



s) = f(x(s), a[v](s)) + a(x(s), a[v](s))v(s), s > 0, 



x(0) = x. 



(3.10) 
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Since I is bounded, it is sufficient to consider the supremum of (I3.9P on 

IMU 2 [0,T-i] < K 



(3.11) 



for some constant K where K does not depend on t, x, a. 

Let x(s) be the solution of (13.1Q[) with the initial condition x(0) = y and set ((s) 
x(s) — x(s). Then, from (A2), we can find constants C\, C% such that 



|C(s)|<|x-y|+ / C 1 |C(r)| + C 2 |C(r)||t;(r)| ?r, 0<s<T. 
Jo 



By Grownwall's inequality, 



|C(s)| < \x-y\ \l+ / (Ci + C 2 |t;(r)|)e^ (Cl+C2KT)l)dr c/r I, < s < T. 



Since we only consider v satisfying (13.111) . we can find Ck > such that 

ICOOI <C K \x-y\, 0<s<T. 

Since l x is bounded, 

l(x(s),a[v](s)) < l(x(s),a[v](s)) + ||^||oo|C(s)l- 

Thus we have 



l(x(s), a[v](s))ds < / l(x(s), a[v](s))ds + L K \x — y\ 
[0,T-t] ' J[o,T-t] 



where Lk = \\I x \\ooCk- Taking the max-plus expectation, 



E, 



Cl.r 



l(x(s), a{v](s))ds 



[0,T-t] 



l(x(s), a[v](s))ds 



+ L K \x - y\. 



U [0,T-t] 

Since a e T(0, T — t) is taken arbitrarily, 

V(t,x) < V(t,y) + L K \x-y\. 

Therefore we have 

\V(t,x) - V(t,y)\ < L K \x -y\, < t < T, x,y e R n . 



(3.12) 



We now show that V(-, x) is uniformly Lipschitz. Let < t < r < T. From (I3.9p . it is 
easy to see that 

V(r,x) < V(t,x). 
So it suffices to show for some M > that 

V(t,x) < V(r,x) + M{r -t). 
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By the DPP for (13.91) and max-plus linearity of the max-plus expectation, 



V{t, x) = inf E+ 
aer(o,r-t) 



l(x(s),a[v](s))ds © V(r,x(r - t)) 



[0,r-t] 



inf Et 

aeV{0,r-t) 



l(x(s), a[v](s))ds 

U[0,r-t] 

By (A2), there exists C > such that for any v G L 2 [0,r — t], 

\x(s) — x\ < Cs + C / \v(r)\dr, < s < r — t. 
Jo 

Since x t— > V(t, x) is uniformly Lipschitz by A3. 12j) . 



E+[V(r,x(r-t))\. (3.13) 



V(r, x(r - t)) < V(r, x) + C K (r - 1) + C 



K 



r-t 



\v(s)\ds, 



where Ck = L K C . Thus, we can estimate the second expectation of (13.131) by 
El[V(r,x(r-t))} 



< 



sup 

v£L 2 [0,r-t] 



V(r,x) + C K (r-t) + C_ 



K 



r-t 



\v(s)\ds — - 



r-t 



\v(s)\ ds 



1 



< V(r, x) + C K (r -t) + -C 2 K (r -t) = V(r, x) +M 1 K (r-t), 
where M\ = C K + (l/2)C/£. Therefore we have from fl3TT31 



V(t,x)< inf E+ 

a£T(0,r-t) 



l(x(s), a[v](s))ds 



U [0,r-t] 

(V(r,x) + M l K (r-t)) 



© (V(r,x) + M x K {r - *)) 



inf Et T 

aeV(0,r-t) 



e 



l(x(s), a[v](s))ds 



U [0,r-t] 



Let us consider the constant strategy a[v](s) = u 

l(x(s), a[v)(s))ds 



inf E+ 

aer(o,t-r) 



[0,r-t] 



<Et x 



l(x°(s),Uo)ds 



[0,t-r] 



(3.14) 



(3.15) 



where x°(s) is the solution of (13. lOf) for a[u](s) = uq. Since l x is bounded, by the same 
argument as (I3.14p . there exists M\ > such that 



E, 



Ox 



l(x°(s), u )ds 



[0,t-r] 

If we take uq G argmin ug ;y l(x, u) 



< l(x,u )+M^(r-t). 



l(x,uo) = minl(x, u) = V(T,x) < V(r,x). 



Thus, we have 



inf E+ x 

aeT(0,t-r) 



l(x(s), a[v](s))ds 

U [0,r-t] 

Hence, if we take = max{M^, M^}, we obtain from (I3.15P 

V(t, x) < V(r, x) +M K (r-t). 



< V(r,x) + M 2 K (r-t). 



□ 
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4 Viscosity approach to value function 



4.1 Nonlinear parabolic equation equivalent to QVI 

As mentioned in Section 2, we want to show that the DPE for (13 .11) is (12. lip . Equation 
(12. lip seems quite reasonable because it is derived from the generator with constant 
control case. One difficulty on (12.111) is very nonlinear, for instance, it is not linear on 
dV/dt. 

On the other hand, by using the idea of [5], it is shown that 02. 11 j) is equivalent to the 
following equation (see Proposition 14. 6p : 

dV 

— + H(x, V(t, x), VV(t, x)) = 0, (t, x) G (0, T) x R n , (4.1) 
at 

where for x G R n , r G R, p G W 1 , 

TC(x,r,p) — min H u (x,p) = min max j (f(x, u) + a(x, u)v ) ■ p — -\v | 2 \ , (4.2) 

u£A(x,r) u£A(x,r) v£~ 



A(x,r) = {u G U ; l(x,u) < r}. 

We define TC(x,r,p) = oo if A(x,r) = 0. Equation (14. ip looks like a standard nonlinear 
equation of parabolic type backward in time. But we need to be careful in 04. IB because the 
Hamiltonian 7i(x,r,p) is discontinuous and it can be oo. As we will see later, since 04. IB 
will be naturally derived by calculating the generator, we mainly discuss (14.11) instead of 
(12. lip . But one should notice that 02. 11B is useful to prove Theorem 12.11 See also Section 
7. We first show that (12.111) and (14. ip are equivalent in viscosity sense. 

Let us recall the definition of viscosity solutions with a discontinuous Hamiltonian. 
For the later argument, we shall define discontinuous viscosity solutions. Let h(£) be a 
function on a subset in a Euclidean space. We denote by h*(£) and the upper and 

the lower semi-continuous envelopes of h, respectively: 

h*(£) — lim sup h{rj), = liminf h(rj). 

Definition 4.1. Let W{t,x) be a locally bounded function on (0, T) x W 1 . W(t,x) is 
a viscosity subsolution (resp. viscosity supers olution) of (14.11) if the following holds: if 
(t, x) G (0, T) x ]R n is a maximum point (resp. minimum point) of W* — ip (resp. W* — ip) 
for a C^function <p(t,x) in (0,T) x ]R n , then 

^(i, x) + n*(x, w% x), v v (i x))>o, 

resp. ^(lx) +nJx,WJi,x),V(p(i,x)) < 
at 

If W(t,x) is a viscosity sub and supersolution, W(t,x) is called a viscosity solution. 

We will give some properties on H(x, r,p) used in |5lj. They can be proved in a similar 
way to [5], so we omit the proofs. 
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Lemma 4.2 (cf. Lemma 2.3, [5]). (i) If r < r' , then A(x,r) C A(x,r'). 
(ii) Let r < r' , x E R n . Then there exists 5 = S(\r' — r\) > such that 

A(y,r) c A(x,r'), \/y E B s (x), 

where Bs(x) is the open ball centered at x with radius 5. 

Lemma 4.3 (cf. Lemma 2.4, [5]). (i) If r < r' , then H(x,r,p) > 7i(x,r',p). 
(ii) Let r < r' , x E R n . Then there exists 5 = 5(\r' — r\) > such that 

H(y,r,p) >H(x,r',p) - L(\p\ + l)\p\\x - y\, Vy E B s (x), Vp G R rt 

/or some constant L > 0. 

Lemma 4.4 (cf. Proposition 2.5, [5]). For x G R™, r G R, p G R n , 

(i) TC*(x,r,p) = H{x,r + 0,p), 

(ii) H*{x,r,p) = H{x,r- 0,p). 

In Lemma 14.41 we can identify the semi-continuous envelopes as follows. 
Lemma 4.5. (i) TC*(x,r,p) = H(x,r,p). 

(ii) TC*(x,r,p) = mi u£ A'(x,r) H u (x,p) where A'(x,r) = {u E U ; l(x,u) < r}. 

Proof. We shall prove (i). Since s i— > 7Y(x, s,p) is non-increasing, 

7i(a;, r + l/n,p) < H(x,r,p). 

Taking n — > oo, 

7i(x,r + 0,p) < H(x,r,p). (4.3) 
To show the equality holds in (14.31) . suppose that 

H(x,r + 0,p) < H(x,r,p). (4.4) 
By noting that s i— > 7i(x, s,p) is non-increasing, we have 

min H u (x,p) = H(x, r + l/n,p) < H{x, r + 0,p), \/n E N. (4.5) 

«£ A(a;,r+l/n) 

Let m„ G v4(x,r + 1/n) be a minimizer of min MG ^( x . r+1 / n ) H u (x,p), i.e., 

H Un (x,p)= min H u (x,p) = Tt(x, r + l/n,p). (4.6) 

udA(x,r+l/n) 

Note that v4(x,r + 1/n) ^ because H(x,r + l/n,p) < oo by (14.41) and (14.51) . Since C/ is 
compact, there exists a subsequence {u nj } and u E U such that 

«ni -»• « (i oo). 

By (USD and (Ojl . 

H Un i(x,p) < H(x,r + 0,p). 
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Thus, taking the limit as j — ► oo, 

H u (x,p) < H{x,r + 0,p). (4.7) 

Since « nj G r + 1/n/), 

l{x,u U] ) <r + l/rij, j = 1,2, ■ • • . 
Taking the limit as j — > oo, we have 

l(x,u)<r, i.e. ,uEA(x,r). (4.8) 



Therefore, we have from (14.71) and (14.8 



H(x,r,p) — min H u (x,p) < H u (x, p) < H(x, r + 0,p). 

u£A(x,r) 

This contradicts to ( 14.41) . Hence we obtain 

H(x, r + 0,p) = TC(x, r,p). 

From Lemma [4.41 (i), we have (i). 

We shall next prove (ii). It is enough to show that 

H(x,r-0,p)= inf H u (x,p). 

Since A(x, r — 1/n) C A'(x, r) (n = 1, 2, • • • ), 

TC(x, r — l/n,p) — min H u (x,p) > inf H u (x,p). 

u£A(x,r—l/n) u£A' (x,r) 



Sending n to oo, 



H{x,r-Q,p)> inf H u {x,p). (4.9) 

ue.A'(x,r) 



Since s ^ H(x, s,p) is non-increasing, 

H(x, r — l/n,p) >H(x,r — 0,p), 
i.e. H u (x,p) > H(x, r — 0,p), \/u G A(x, r — 1/n). (4.10) 

Note that 

oo 

A'(x,r) = (J A(x,r - 1/n). 

n=l 

So, we have from (14.101) 

H u (x,p) > H(x,r - 0,p), WueA'(x,r). 

Thus, we have 

inf H u (x,p) > H(x,r -0,p). □ 

udA'(x,r) 

By using Lemma 14.5^ we can prove the following result on the relation between (12. lip 
and (Oil . 
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Proposition 4.6. Let W(t,x) be a locally bounded function on (0, T) x M. n . W(t,x) is a 
viscosity subsolution ( resp. viscosity supersolution) of ( 12, lip if and only ifW(t, x) is a vis- 
cosity subsolution ( resp. viscosity supersolution) of (14. ip . Here W(t, x) is a viscosity sub- 
solution ('resp. viscosity supersolution) of (12.111) if the following holds: if (i,x) G (0, T) x 
W 1 is a maximum ( resp. minimum) point of W*(t, x) — ip(t, x) ( resp. W*(t, x) — ip(t, x) ) 
for a C 1 -function <f(t,x) on (0,T) x R n , then 

rmrimax|^(t,x) + H u (x, V(f{t, x)), l{x, u) - W(t,x)| > 

resp. minmax {^(t, x) + H u (x,Vip(t, x)),l(x, u) - WAi,x)\ < ] . 
ueu [ at J / 

Proof. Note that we may assume W*(t, x) = (p(t,x) in the definitions of subsolutions 
without loss of generality. Then, it is sufficient for the equivalence of subsolutions to 
show that for a C 1 -function <p(t,x) and (t,x) G (0,T) x R™, the following inequalities are 
equivalent: 

minmax \^-(i,x) + H u (x, V(p(t,x)),l(x,u) -<p(i,x)\ > 0, (4.11) 

u&U [ Ot J 

^(t, x) + H*(x, V (t, x), V<p(i, x)) > 0. (4.12) 
Let us assume (14.111) holds. Then, we have 

maxj^(t,x) + H u (x, V<p(i,x)),l(x,u) -</?(£,£) J > 0, Wu G U. (4.13) 

If u G A'(x, <p(t, x)), i.e., l(x,u) — (p(t,x) < 0, (14.131) implies that 

^(i,x) + H u (x,V^i,x)) >0. 

Therefore, 

^-(i, x) + H u (x, Vip(i, x)) > 0, Vm G A'(x, <p(i, x)). 



Thus, we have 



^(t,x)+ inf H u (x,V<p(i,x))>0. 

Ot u€A'(x,<p(i,x)) 



By Lemma [4.51 (ii), we have 

^(i,x)+W(x,tp(t,x),V<p(i,x)) > 0. 
We suppose ( 14. 12ft holds. From ( 14. 12[) and Lemma 1431 



, (t,x)+ inf H u (x,V<p(t,x)) > 0. (4.14) 

Ot ueA'(x,<fi(t,x)) 
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If A'(x,<p(i,x)) = V>, 

l(x, u) > <f(t, £), Vtt G U. 

Thus we have 

max | + H u (x, Vtp(i, x)), l(x, u) — cp(i, x)\ > l(x, u) — ip(i,x) > 0, Vw G U, 
which implies 

minmax < -7r~(i, x) + H u (x, V<f(i, x)), Z(x, u) — ip(i, x) 1 > 0. 

«et/ y ot J 

If A'(x,ip(i,x)) ^ 0, we have from (Oil) . 

^(t, x) + #"(x, V(/?(£, x)) > 0, Vw G A'(x, p(t, x)). (4.15) 
at 

Let u E U be arbitrarily taken. If l(x,u) < ip(t,x), i.e., u G A'(x, x)), then (14. 15j) 
implies 

maxi^(£,x) + H u (x, V<p(i,x)),l(x,u) -<p(i,x) \ = ^(i, x) + H u (x, V<p(t, x)) > 0. 
[at Jot 

On the other hand, if l(x,u) > tp(t,x), 

max | "7^"(£> + H u (x, Vip(i, x)), Z(x, it) — </?(t, x) 1 > i(x, w) — cp(i, x) > 0. 



Thus, 
Therefore, we have 



max <{ ^r(i%) + H u (x, Vy?(t, x)), Z(x, w) -<p(t,x) } > 0. V// G ( r . 



minmax |"^"(*> ^) + H u (x, Vip(i, x)), Z(x, w) — <p(t, x) ^ > 0. 

For the definitions of supersolutions, we may also suppose W*{t,x) = (p(t,x). Thus, 
in the proof of the equivalence of supersolutions, it suffices to check that the following 
inequalities are equivalent: 

minmax \ x) + H u (x,V(p(i,x)),l(x,u) -<p(t,x)\ < 0, (4.16) 

u&u ^ at J 

^(ix)+H*(x,<p(i,x),V<p(t,x)) <0. (4.17) 
Suppose ( 14. 161) holds. Since U is compact, there exists u & U such that 
max |"^~(^' ^0 + H u (x, S/ip(t, x)), Z(x, u) — <p(t, x) 
= minmax |"^"(^> ^) + H u (x, V x)), /(x, u) — ip(i, x) }> < 0. 
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Thus, 

^(i,x) + H a (x,Vip(i,x)) <0 and l(x, u) - <p(i, x) < 0. 
Since l(x,u) < cp(t,x), we see u G A(x,(p(t,x)). Therefore we have 

^(ix)+H&<p(i,x),V?(ix)) = ^(ix)+ min H u (x, V<p(i, x)) 
at at ueA(&,ip(t,x)) 

< ^(t,x)+H u (x,V^(i,x))<0. 

By Lemma [4.51 (i), we obtain 

dip ~ a 

— (i, x) + TL*(x, (p(t, x), V<p(t, x)) < 0. 

We assume (14.171) holds. From ( 14. 17ft and Lemma [4.51 (i). 

^£(t,x)+ min H u (x, Vip(i,x)) < 0. (4.18) 

ueA(x,<p(i,x)) 

Note that A(x, <£>(i, x)) ^ because the minimum of ( 14.181) is finite. Taking a minimum 
point u G A(x,<p(i,x)) in (14.180 . 

^(t,x) + fT(x,V^(t,x)) <0 

Combining w G <£>(£, x)), i.e., /(x, u) — x) < 0, 

max j^(t,x) +H u (x, V(p(i,x)),l(x,u) - <p(i, x) j < 0. 

Thus, we obtain 

min max j^(£, x) + (x, V(p(i,x)), l(x,u) - (p(i,x) \ < 0. □ 
u&j [at J 

4.2 Generators and evolution equations 

In optimal control problems, DPPs can be often described by the semigroups associated 
with the value functions. Once the control problem is related to the semigroup and the 
generator is identified, one can prove that the value function is a viscosity solution of the 
evolution equation with the generator (cf. [13]). We shall discuss our max-plus control 
problem by the semigroup-generator approach and prove that the value function (13.11) is 
a unique viscosity solution of (14. ip with (I2.12p . 

For < t < r < T, we define operator F t r acting on <p : W 1 — > Mr by 



F t>r <f>(x) = inf E+ 



l(x(s),a[v](s))ds@<f>(x(r)) 



x G 
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We understand that F t)t is an identity operator. By using F t r , the value function V(t, x) 
can be written as 

V(t,x)=F ttT 0(x), 

where is a constant function taking its value = — oo. Note that = — oo is the additive 
identity in max-plus algebra. Then, (13.21) can be written in terms of F tiT as follows: 

F t , T = F t , r F r , T 0, < t < r < T. 

The DPP of (13.11) given by (13. 2\\ corresponds to the semigroup property of {F^ r }. 

Since V(t, x) involves an inf-sup as seen in (13.11) . the form of the generator of {F t r } is 
not obvious. Moreover, it is expected that the form depends on the choice of strategies 
classes (cf. [16]). By following the argument used in [IB] , we can show that our strategy 
class T(t,T) satisfying (SI) and (S2) is related to TC(x,r,p) in (14.11) . To state the result 
on the generators, we denote by ((0, T) x IR n ) the set of bounded C 1 -functions with 
bounded first order derivatives. 

Theorem 4.7. For any <p G C£((0,T) x W 1 ) and (t,x) G (0,T) x « n , we have 

limsupi{F M+5 ^(t + 5,-)(x) -<p(t,x)} < +W(x,<p(t,x),V<p(t,x)), (4.19) 

5-+0+ o at 

^-(t,x)+H*(x,<p(t,x),Vip(t,x)) < liminf \{F t>t+s <p(t + 8, -)(x) -<p(t,x)}. (4.20) 
at 5->o+ o 

We need a result on approximations of strategies in T(t, T). The proof is given in the 
Appendix. Let Aek^, T) be the set of Elliott-Kalton strategies from L°°([t,T]]U) into 
L%T}. 

Lemma 4.8. Let a G T(t,T). For any (3 G A^(t,T) and any e > 0, there exist 
u e G L°°([t,T];U) and if G L 2 [t,T] such that 

\a[v £ }{s) -u £ {s)\ < e, Vs G [t, T) and f3[u e }{s) = v e {s) a.e.se[t,T}. 

Proof of Theorem 14.71 We first prove (I4.19p . In Lemma 14.51 (ii) with r = ip(t,x), 
p = V<p(t,x), we consider u G U such that 

l(x, u) < <f(t, x). 

It is enough to prove that 

limsupi{F M+ ^(t + 5,-)(x) -<p(t,x)} < ^ r (t,x) + H u (x,Vif(t,x)). (4.21) 
5^0+ o at 

We take po > such that 

l(x,u)<tp(t,x)-p . (4.22) 
If we take a constant strategy a[v](s) = u for F t;t+ g(p(t + 8, -)(x), 
F t , t+S <p(t + 8, -)(x) 



l(x(s), u)ds © (f(t + 5, x(t + 5)) 

[t,t+S] 



sup < l(x(s),u)ds®<p(t + 5,x(t + 5))-- I \v(s)\ 2 ds}. (4.23) 

v£L 2 [t,t+S] {J[t,t+6] 2 



t+8 
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Since I and <p are bounded, the supremum in (14.231) can be restricted to those v satisfying 

•t+6 

\v(s)\ 2 ds<M, (4.24) 

for some M > 0. Here M does not depend on u and 8. Then, we can show that there 
exists c(M) > such that for any v satisfying (I4.24p . 

\x(s) -x\< c(M)8 1/2 , t<s<t + 8, (4.25) 
\ip(t + 5, x(t + 5))- cp(t, x) | < c(M)8 1/2 , 
\l(x(s),u) -l(x,u)\ < c(M)8 l/2 , t<s<t + 8. 

For small 8 > 0, we have 

\<p(t + 8,x(t + 8)) -<p(t,x)\ < -p , 
\l(x(s), u) — l(x, u)\ < -po, t < s < t + 8. 
Then, by using (I4.22p . we have for s e [t, t + 8] 

l(x(s), u) < l(x, u) + ^p < <f(t, x) - < v{t + 8, x(t + 8)), 

which implies 

l(x(s),u)ds < <p(t + 8, x(t + 8)) 



' [t,t+S] 

for small 8 > 0. Thus, from fl423|) with fl424j) . 
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F t:t+S (p(t+5, -)(x)— <p(t, x) < sup ^ (p(t + 8,x(t + 8)) — ip(t,x) — ^ / \v(s)\ 2 ds 

'i 2 [t,t+5] 



\\v\\ 2 ,, r <M (. 2 j t 



t+5 



for small 5 > 0. By using the fundamental theorem of calculus and (14.251) . 



1 



t+8 



<p(t + 6,x(t + 8))-<p(t,x)-^ J \v(s)\ 2 ds 

j), u) + a(x(s),u)v(s)) ■ V<p(s, x(s)) - ^\v(s 
(t, x) + (f(x, u) + a(x, u)v(s)) ■ V<p(i, x) — -|t>(s)| 2 <is + o(8) 

(x,u)v) ■ V(p(t,x) - ^\v\ 2 
where o(5) is uniform on v satisfying (I4.24H . Therefore we obtain 

F ttt+S <f{t + 8, -)(x) - (p(t, x) < (j£(t, x) + H u {x, V<p{t, x)))6 + o(6). 
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rt+S 




Jt 


ds 


rt+S 


dip 


Jt 


dt [ 


(dip 


(t,x 


1 dt 



This implies (I4.2ip . Hence we have proved (14.191) . 

To prove (14.201) . we use the idea of the proof of [HH Proposition 2.6]. If u G L°°([t, t + 
5); U) and v G L 2 [t, t + 5] are given, let us define F^ +S (j){x) for <\> : R n -> R~ by 



1 



t+s 



OW= / l(x(s),u(s))ds®(j)(x(t + 5))-- \v(s)\ 2 ds, 

J[t,t+5] A Jt 

where x(s) is the solution of ( 12. 2ft . 

We consider a particular (3 G A^(t, t + 5): 

/3[w](s) = (t(x(s),m(s)) t Vv?(s,x(s)), t < s < t + 5, 
where x(s) is the solution of (12. 2p with v(s) = /3[u)(s) 
Fix p > 0. We shall show that for small 5 > 

^f;V(H5,-)(x)- V (M) 

> min | ^(t, x) + W(ar, <p(t, x) + p, V<p{t, x))J 6 + o(S), P - - Msj , (4.26) 

where o(5) and M are uniform on u G L°°([t, t + S];U). 

For the first case, suppose u(s) G A(x,<p(t,x) + p) for a.e. s G [t, i + 5]. From the 
definition of TC(x, <p(t, x) + p, V<p(i, x)), 

H(x, <p(t, x) + p, V<p(t, x)) 

< sup I (/(x, m(s)) + cr(x, -u(s))f ) • V<p(£, x) — - \v\ 2 



= (f(x,u(s)) + a(x,u(s))a(x,u(s)) T V(p(t,x)) ■ V<p(t, x) 

— -|er(x, -u(s)) T V<p(£, x)| 2 , a.e. s G [<,£ + 5]. 

Integrating the above inequality on [t, t + 5], we have 
H(x, <p(t, x) + p, V<p(i, x))8 

< jf" |(/(x, M (s)) + o-(s, M (s))(T(x,M(s)) T V<p(t,x)) • V^>(t,x) 

— -|a(x, -u(s)) T V<p()!;, x)| 2 |<is. 

Since /, a, V<p are bounded and continuous, there exists K > 0, which does not depend 
on u(-), such that 

|x(s) -x| < K(s-t), t < s < t + S. (4.27) 

Thus, we have 

H(x, ip(t, x) + p, ip(t, x))8 

< + <r(z(s),u(s))0M(s)) • V<p(s,x(s)) 

- \\Ku]{s)\ 2 \ds + o{5). (4.28) 
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where o(5) is uniform on u(-). 
By ( 12.21) for x(s), we have 

i r t+s - 

<p(t + 6,x{t + S))-<p(t,x)-- j \/3[u](s)\ 2 ds 

t+S ^(s,x(s)) + (f(x(s),u(s)) + a(x(s),u(s))P[u](s)) ■ V<p(s,x(s)) - ±0[u](s)\ 2 ds 
\(t 

~ 2 

where o(S) is uniform on u(-). Thus, we have from (I4.28|) 

i r t+s 

<p(t + 5,x(t + 8))-<p(t,x)-- / 0[u}(s)\ 2 ds 



x)5 + f <j (f(x(s), u(s)) + a(x(s),u(s))f3[u}(s)) ■ Vp(s, x(s)) 

hp[u](s)Ads + o(S), 



> [ ^(t,x)+H(x,<p(t,x)+p,V<p(t,x)) ) <) + <>(<)). 



Therefore, we obtain 



F^Mt + 6,-)(x)-v(t,x) 



l(x(s),u(s))ds e <p{t + 5,x(t + 5)) - - / \(3[u](s)\ 2 ds - <p(t,x 



t+s 



[t,t+s] 2 j t 

1 r t+s 

> (p{t + 5, x(t + 6))- (p(t, x)-- \j3[u] (s)\ 2 ds 

2 Jt 

> (^-(t, x) + H(x, <p(t, x) + p, V<p(t, x))^j 5 + o(5). (4.29) 

For the second case, we suppose there exists a subset Is C [t, t + 5] with positive 
Lebesgue measure such that u(s) £" A(x,<p(t,x) + p) for s £ I§, i.e., 

l(x, u(s)) > ip(t, x) + p, Vs £ 1$. 

Since I and (p are Lipschitz, 

/(?/, w(s)) > <p(r, z) + p — L(\y — x\ + \z — x\ + 6) for s £ Is, t < r < t + 5. 
Then, by $Q7|) , we have 

Z(£(s), u(s)) > <p(r, x(r)) + |, s £ J 5 , i < r < i + 5 

for small 5 > 0. Therefore we have 

u(s))cis > (p(r, x(r)) + — , i < r < t + <5. 

[i,t+<5] ' 2 
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By using this, 

F t,t+s ] f( t + 5 r)(x) -<p(t,x) = I l(x(s),u(s))ds®ip(t + 5,x(t + 5)) 



[t,t+S] 

1 

~ 2 



t+8 

\p[u}(s)\ 2 ds - <p(t,x) 



1 



> / l(x(s),u(s))ds-- \p[u](s)\*ds-<p(t,x) 



[t,t+s] 2 j t 



t+8 



p 1 



t+8 



><p(t,x) + Z-^ J \f3[u}(s)\ 2 ds-<p(t,x) 

>^-M5. (4.30) 

for some constant M > 0. Here we used /3[it](s) = a(x(s), u(s)) T Vy?(s, x(s)) is bounded. 
Hence, by and (O0I) . we obtain (TOED . 

Now we shall prove (14.201) . We first fix e > 0. Let a G T(t, t + 5) be taken arbitrarily. 
It implies from Lemma I4T81 that there exist u e ' S G L°°([t, £ + 5]; U) and f 6 ' 5 G L 2 [t, t + 5] 
such that 

\a[v e ' s ](s) -u e ' s (s)\ < e5, Vs G [t,t + 5) and /3[u e ' 5 ](s) = v e ' s (s) a.e. s G [t,t + 5]. (4.31) 

Let a; e ' ,5 (s) (resp. x e ' s (s)) be the solution of (12.21) for u(s) = a[v e ' s ](s) and v(s) = v t,s (s) 
(resp.u(s) = u e ' 5 (s) and v(s) = j3[u e,s ](s)). By using (14.311) . we can show that 

\x e ' s (s) - x e ' s (s)\ < Ce5 2 , t<s<t + 5 

for some C > 0. Then, it is not difficult to see that 

l{x £ ' s {s), a[v e ' s ](s))ds © (p(t + 5, x e '\t + 5)) 

[t,t+8] 

> / l(x e ' S (s),u e ' S (s))ds®if(t + 5,x e ' S (t + 5))-Ce5-Ce5 2 

J[t,t+8] 

for some constant C. Since v e ' 5 = j3[u e ' 5 ], 

Ftt+s^ W + ■)(*) - > F^f ue \(t + 5, -)(x) - ip(t,x) - Ce5 - Ce5 2 . 

By 0426]), 

F^8 ] '^^t + S,-)(x)-^t,x) 

> min | (j£(t, x) + H(x, <p(t, x) + p, V<p(t, x))^j 5 + o(5), ~ - M«J J - (7e5 - (7e<5 2 . 

Note that o(5) is uniform on a because (14.261) is uniform on u. Therefore we obtain 
F t} t+sf(t + 5, -)(x) - ip(t,x) 

> min I (^(t, x) + H(x, <p(t, x) + p, V<p(t, x))j 5 + o(5), | - M<J j - Ce5 - Ce5 2 . 
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Dividing the both side by S > and taking liminf as 5 — ► 0+, 

1 dip 
liminf - {F t t+S (p(t + 5, -)(x) - <p(t,x)} > -wr(t,x) + H{x,(p(t,x) + p, Vcp(t, x)) - Ce. 

8^0+ o Ot 

Then, by sending e — > 0, we have 

1 d 
liminf -{F t t+S (p(t + 6, -)(x) -(p(t,x)} > -^r(t,x) + H(x,(p(t,x) + p, Vtp(t, x)). 



By Lemma S3 (i), if we take p — > 0+, we finally have (14.201) . □ 

Under the DPP and the estimates on the generator, we can easily characterize V(t,x) 
as a unique viscosity solution. The uniqueness is implied by the comparison theorem in 
the next section. 

Theorem 4.9. V(t, x) is the unique bounded Lipschitz continuous viscosity solution of 
(14.11) ( equivalently (12.111) ) with the terminal condition (12.121) . 

Proof. First of all, we point out that we may suppose the smooth test function ip(t, x) in 
Definition is taken from C^((0, T) x R n ) because V(t, x) is bounded and the definition 
of viscosity sub/supersolutions only uses a local information of W(t, x) — (p(t, x). 

We only show that V(t, x) is a subsolution of (14.11) since the supersolution part is 
proved in a similar way. Let (t, x) G (0, T) x R n be a maximum point of V(t, x) — tp(t, x) 
for if G Cl((0,T) x R n ) and V{t,x) = tp(i,x). If we use (Q at (t,x) = (t,x), then 



V(i, x) = inf E 
aer{i,i+s) 



ix 



[U+6] 



l(x(s),a[v](s))ds © V(i +S,x(i + 6)) 



Since V(i,x) = <p(i,x) and V(t,x) < <p(t,x) for any (t,x) G (0,T) x R n , 



(pit, x) < inf E 



tx 



l{x(s), a[v](s))ds © ip(i + 5, x(i + 5)) 



U [t,t+S] 



By using the notation F ttt+ $, this inequality can be written as 

<p(i, x) < F ii+sV (t + 5, -)(x). 



Thus, we have 



From Theorem 14.71 



< limswp -{F {{+S ip(i + 5, -)(x) -<p(t,x)}. 



< ^(t, x) + H*(x, tp(t, x), V<p(t, x)). 



Therefore V(t,x) is a viscosity subsolution of (14.11) . 



The uniqueness is a consequence of Theorem 15.11 in the next section. □ 
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4.3 Remarks on other strategy classes 

To discuss the max-plus control problems, one might want to choose a different class of 
strategies. Actually, there exist classes for which the generator-DPP arguments work. We 
shall make remarks on other possibilities for strategy classes without details. 

First of all, one might think that Elliott-Kalton strategy class should be a natural can- 
didate for the problem. Let V E K(t,x) be the value function of (13.11) defined by T E K(t, T) 
instead of T(t,T). It is not difficult to see V E R(t, x) satisfies the DPP in a similar way to 
the case of T(t, T): for t < r < T, x G W l , 



V EK (t,x) = inf E+ 

a&V EK {t,r) 



l{x{s), a[v\(s))ds © V E k{t, x(r)) 



U[t,r] 



For t < r, denote by F t E r K the operator associated with F E K(t, r): 



F t E r K <pi.O 



inf E tx 

a&T EK {t,r) 



[t,r] 



l(x(s),a[v](s))ds © (f>{x(r)) 



If we calculate the generator, we can prove that for ip G C£((0, T) x M n ) and (t,x) G 
(0,T)xM n , 

lim sup ~ {F t %ip{t + 5, -)(x) - ipit, x) }<^-(t,x) + K?(x, <p(t, x), V<p(t, x)), (4.32) 

5^0+ Ol 



dt 
where 



^(t, x) + K«{x, <p(t, x), V<p(t, x)) < lim inf - {F t % V (t + 5, -)(x) - <p(t, x)} , (4.33) 

' " S— >0+ 



fC(x, r, p) = max min I (f(x, u) + a(x, u)v) • p \v l2 

' ' ueA(x,r) | ' 2 



(4.34) 



See Appendix IB1 for the proof. Thus, Ver^, x) can be shown to be a bounded Lipschitz 
continuous viscosity solution of 



dV, 



EK 



Of 



(t, x) + JC(x, V EK (t, x), VV EK (t, x)) = 0, (t, x) e (0, T) x 



(4.35) 



with the terminal condition (I2.12p . Moreover, V E R(t, x) can be shown to be unique in 
such function class since we can have a comparison theorem for (14.351) by using similar 
ideas to Theorem 15.11 Here we note that JC(x,r,p) < 7i(x,r,p) and they do not coincide 
in general. Since our DPE is (12.11)) . equivalently (14.11) . using T EK {t,T) leads to a wrong 
equation. 

There can be another class associated with (14.11) . a G r E x(t,T) is called a strictly 
progressive strategy if for any (3 G A E x{t, T), there exist u G L°°([t, T];U) and v G L 2 [t,T] 
such that 

a[v] = u, f3[u] = v a.e.on [t,T], 

where A EK (t, T) is the set of Elliott-Kalton strategies from L°°([t, T]; U) into L 2 [t, T] (see 
[TO] . [T3] and [16]). If we denote by Tsp{t,T) the set of strictly progressive strategies on 
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[t,T] and Vsp(t,x) is the value function given by T S p{t,T), it is also seen that Vsp(t,x) 
satisfies the DPP: 



V SP {t,x) 



inf E tx 

aer SP (t,r) 



/(x(s), a[v](s))ds © V S p(r, x(r)) 



[t,r] 



for t < r < T and x G M n (cf. [16] for max-plus multiplicative running cost). Let the 
operator corresponding to T S p(t,r) be 



if r p 0(x) = inf E+ 
aer SP (t,r) 



l(x(s), a[v](s))ds © (f>(x(r)) 



U[t,r] 



We can show that the generator of Fff is associated with H by the exactly same way as 
T(t, r) without using Lemma 14. 8t 

limsup - {F t s t p +5 ip(t + 5, -)(x) - <p(t, x)} < ^(t, x) + p(t, x), V<p(t, x)), 

8^o+ o at 

<¥ (t,x) +H*(x,<p(t,x),V<p(t,x)) < liminf \ {F t ^ s <p(t + 6,-)(x) -<p(t,x)} 



for ip G ^((0, T) x R n ) and (t, x) G (0, T) x M n . Therefore, K SP (t, x) is the unique bounded 
Lipschitz continuous viscosity solution of (14.11) with (12.121) . Hence, Vsp(t,x) = V(t,x). 
But we note that it is not clear that Markov control policies are strictly progressive in 
general. Thus, T(t, T) may be a better class if we consider the connection to the verification 
theorem. See also [121 Remark XI.9.1] for the discussion on Markov control policies of 
a differential game problem. Lemma 14.81 states that a property like strict progressivity 
holds approximately, if a G T(t,T). This approximation property is useful in deriving 
Theorem 14.71 instead of directly using strict progressivity. 

We have not made use of the following differential game interpretation of the max- 
plus control problem. In this interpretation, u(s) is a minimizing control and v(s) is a 
maximizing control. The differential game dynamics are (I2.2|) and the game payoff is 



P(t, x; u, v) 



l(x(s), u(s))ds 



[t,T] 



\v(s)\ 2 ds. 



According to ( 13.1 ft 



V(t,x) 



inf sup P(t,x:a\v],v). 
aev(t,T) veL2%T] 



(4.36) 



(4.37) 



In the Elliott-Kalton definition of lower differential game value, T(t,T) is replaced by 
^EK{t,T). Hence we should not expect V(t, x) to agree in all cases with the Elliott- 
Kalton lower value. 



The right side of (14.361) involves both a max-plus integral and an ordinary integral. For 
this reason we could not use standard differential game arguments to obtain a dynamic 
programming principle. Instead, properties of max-plus conditional expectations were 
used to prove the dynamic programming principle in Theorem 13 .31 
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A different kind of differential game payoff is 

Pi(t,x;u,v)= [ l(x(s),u(s)) -\ [ \v(s)\ 2 ds. (4.38) 

This is the case of "max-plus multiplicative running cost" mentioned in the Introduction. 
For this payoff, standard differential game methods can be used. Let 

Vx(t, x) = inf sup P%(t, x; a[v ], v). (4.39) 

a€T SP (t,T) veL 2[ tjT] 

Then V\(t, x) agrees with the upper Elliott-Kalton value for the differential game (not the 
lower value). See [131 P- 393]. 



5 Comparison theorem 

In the proof of Theorem I4.9[ we used the uniqueness of bounded Lipschitz continuous 
viscosity solutions of (14. II) with (12.121) . This is a consequence of a comparison theorem 
stated in the present section. So far, we only consider continuous viscosity solutions. We 
prove a comparison theorem for semi-continuous cases because we need it in the argument 
for risk-sensitive limits. 

As we noted, the Hamiltonian 7i(x, r,p) is not sufficiently regular to apply the general 
comparison results of viscosity solutions. Following the ideas used in the proof of [51 
Theorem 4.2] with Lemmas I4.2H4.41 we will get a weak comparison theorem for (14.11) . 
The statement of the weak comparison theorem is similar to [TBI Theorems 8.1 and 8.2] 
by assuming a Lipschitz viscosity solution. 

Theorem 5.1. Let W(t,x) (resp. W(t, x)) be a bounded function on (0, T] x W 1 and a 
viscosity subsolution (resp. viscosity super solution) of (14. ip . Assume that x i— > W*(T, x) 
and x I— > W*(T, x) are continuous and there exists a bounded continuous viscosity solution 
U G C((0,T] x W 1 ) of (14.11) such that x \— > U(t,x) is Lipschitz continuous uniformly on 
(0,T) and_W*(T,x) < U(T,x) < W*(T,x) for x G W^. Then W*(t,x) < U(t,x) and 
U(t,x) <W*(t,x) on (0,T] x W 1 . Hence, W*(t,x) <W*(t,x) on (0,T] x R n . 

Proof. We shall only show U(t,x) < W*(t,x). W*(t,x) < U(t,x) can be proved in a 
similar way if we change the role of U{t,x) (resp. W*(t,x)) to W*(t,x) (resp.U(t,x)). 
Let < 9 < 1 and (3 > 0. We choose a smooth nonnegative function g : MJ 1 — > M such 
that 



W*(t,x) < g(x), V(t,x) G (0,T] x R n , (5.1) 
llVpHoo < oo, g(x) — > oo (\x\ — > oo). (5.2) 



Define W 6 ^ : (0, T] x R n -> R by 



W e '\t, x) = (l- 9)W*(t, x) + 6g(x) + L6(T -t) + j, 

L 
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where L > is a constant. 

We can show that there exists a constant L > depending on ||/||oo, Halloo, ||Vg||oo 
such that 

— — + H*(x,W 9 ' l3 (t,x),VW d, \t,x)) < ~ in (0,T) x R n (viscosity sense), (5.3) 
at t 2 

W* < W 9 ' 13 in (0,T] x R n . (5.4) 

These can be seen if W(t, x) is a classical supersolution in the following way. The viscosity 
case can be also verified by using the same kind of arguments. 

By L > 0, 8 > and (jOl . 

W e '\t, x) = (1 - 0)W(t, x) + 0#(x) + L6(T -t) + - 

> (l-6)W(t,x) + 6g(x) 

> (1 - #)W(t, x) + flW^t, x) = W(t, x). 

Thus we have (15.41) . 

We shall show (15. 3p . Since p i— > H u (x,p) is convex, 

fP(x, VF 9,/, (t, x)) < (1 - 0)# u (x, Vf(t, x)) + 0iP(x, V#(x)). 

By noting that /, a and Vg are bounded, there exists C > depending only on 
IMIoo, || V^Hoo such that 



H u (x, VW 6 '\t, x))<{\- 6)H u {x, VW{t, x)) + 6C. 



Then, we have 



<)] 1 + H u (x, VW 9 '\t, x)) < (1 - 0) f 9 ^- + H u (x, VW(t, x))) -(L-C)e-^. 



dt V ' w // - v , y m v , jjj v y t2 

If we take L > such that L > C, 

+ iT (x, VF 9 '^(t, x)) <(l-6)(j?- + H u (x, VF(t, x))) - £ 
Taking the minimum over it G A(x, x)), we obtain 

^t— + mm H u {x,VW e,l3 {t,x)) 

Ot ueA(x,W(t,x)) 

fdW \ 8 

- {l ~ 9) \dt + U{X ' W{t ' X) ' W(t ' X)) ) ~ £ 

- t 2 
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In the last line, we used that W(t,x) is a supersolution of (14. II) . Since W(t,x) < 
W ' (t,x), we have A(x, W(t, x)) C A(x, W ' (t,x)). Thus, we have 

H(x,W ' l3 (t,x),VW $ ' (t,x))< mm H u (x,VW 9 '\t,x)) 

u&A(x,W(t,x)) 



Therefore we obtain (I5.3P 
>rov 

by 



We shall prove U < W 6 ^ in (0,T] x W 1 . For e > 0, we define $ e : (0,T] x (0,T] x 

$ e (t, s, x,y) = U (t, x) - y) - —\t - s\ 2 - —\x - y\ 2 . 

By using standard arguments in viscosity theory, it is not difficult to see that for sufficiently 
small e > 0, there exists (t e , s e , x e , y € ) E (0, T] x (0, T] x R n x W n such that 

$ e (t e , s e , x e , y e ) = sup $ e (t,s,x,y), (5.5) 
t,se(o,T] 



OgiVe) + - + Ute - s e \ 2 + hx e - y e \ 2 < M, (5.6) 
s e 2e 2e 

where M > is a constant independent of e > 0. Again, by standard arguments, we can 
show that 

Y e \te ~s e \ 2 + ^\x e - y e \ 2 - (e -> 0). (5.7) 

Now we are ready to prove U < W in (0,T] x R n , equivalently, 

max (U - W 9 ^) < 0. 

(0,T]xR n 

On the contrary, we suppose that 

max (U - W°' P ) > 0. (5.8) 

(0,T]xK™ 

Under the assumption (I5.8p . it can be seen that 

< t e , s t < T for sufficiently small e > 0. (5.9) 
If ( 15.91) does not hold, there exists a sequence {en}^ = i (e n I 0) such that for each n = 

te n =T or s €n = T. 
Since \t e — s e \ — > as e — > by (15. 6p . 

te n ,S £n (n ->• OO). 

As for the asymptotics on x e , y e through e = e„, we can see from (15.61) that there exists 
iGl™ such that 

x e n ^Ue n — ¥ x (n — > oo) by taking a subsequence of {e n }. 
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We estimate U(t en ,x tn ) — W ' (s en ,y en ) from the above. In the case where t tn = T, 

U(t tn ,x tn )-W*' f '(a em ,yJ = U(T,x tn )-W fiJ '(8 em ,yJ 

= U{T,x tn )-W°*{T,x tn ) +W e ' f3 (T,x en )-W 9 '\s en ,y en ) 
<W 9 '\T,x €n )-W e '\s en ,y en ). 

In the last line, we used U < W * < W on {t = T} x M. n . In the case where s en = T, 

U^xJ-^is^yJ = U(t en ,x en )-W^(T,y en ) 

= U{t en , x en ) - U{T, yj + U(T, yj - W°'\t, y e J 
<U(t en: xJ-U(T,y £n ). 

Similarly, we used U < W 9 ' 13 on {t = T} x R". Therefore, we have 

U(t tn ,xJ-W°' /, {8 en ,y en ) < u,^{W 6 '\T,xJ-W e '\s en ,y en ),U{t en ,x en ) - U(T,yJ}. 
Taking limsup as e n — > 0, 

\imswp(U(t en ,x en ) - W ' (s en ,y en )) 

< limsupmax{W^(T,x £ J - W'^yJ, L^sJ - U(T,y e J} 

e n - + 

< max{limsup(F^(l>eJ - ^(s en ,|/ en )),limsup(C/(t e „,x e J - U(T,yJ)} 

Noting that x t— > W (T,x) is continuous, W ' is lower semi- continuous and U(t, x) is 
continuous, we have 

limsup(E7 {t tn , x tn ) - W*^^, y €n )) < 0. (5.10) 

On the other hand, from (|5.8p . 

0< max (U -Vf fi ) < max $Jt, s, x, y) < U(t e , x e ) - F 0,/? (s e , yX 

(0,T]xK n t,s6(0,T] 

Thus, we have 

< max (U - W^) < liminf(^(t £n , x e J - W 9 '\s €n , y e J), 

(0,T]xR™ «n^0 

which contradicts to (I5.10p . Therefore, (I5.9P has to hold. 

For sufficiently small e > 0, it implies from (15.91) that (t e , s e , x e , y t ) is a maximum point 
of $ e (t, s,x,y) in (0, T) x (0, T) x M n x M n . In particular, since (t e ,x e ) is a maximum 
point of $ e (i, s e , x, y e ) and U(t,x) is a viscosity subsolution, we have 

^(t e -s £ )+H*(x e ,U(t e ,x e ),^(x e -y e )) >0. (5.11) 
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In a similar way, since (s e , y e ) is a minimum point of — $ e (i e , s, x e , y) and is a viscosity 
supersolution of (15.31) . 



-{t t - s e ) + H*(y £ , W 9 '\s € , y e ), -(x e - y e )) < -4 



(5.12) 



Subtracting (I5.12p from (15. lip , we have 

H*(x e ,U(t e , x e ),-(x e - y e )) -H 3f {y e ,W 6 ^{s e ,y e ),-{x e -y e )) > ^. 



(5.13) 



Let us take a > such that 



< a < max (U - W^) 



Since max( ,T]xi»(^ — W ' ) < U(t e ,x e ) — W ' (s e ,y e ) for e > 0, 



a < U(t e ,x e ) - W 9 '\s e ,y e ), Ve > 0. 



We choose r\ < r 2 such that 



W ' (s £ , y e ) < r\ < r 2 < U (t £ , x e ) and r\ — r\ = a. 



By Lemma [4.31 (i) and Lemma [4.41 (ii), 

TC*(x e ,U(t e ,x e ), -{x e - y € )) = TC(x e ,U(t e ,x e ) - 0, -(x e - y € )) 
e e 

< H(x e ,r e 2 , -(x e -y e ))- 

By Lemma [4.31 (i) and Lemma [4.41 (i), 

H*(y e , W* ,/3 (s £ ,y £ ), -(x e - y e )) = H(y t1 W e '\s e) y e ) + 0, ~{x e - y e )) 
e e 

> n(ye,ri,~(x e -y e )). 

Thus, we have from (15.131) 

H{x e , r e 2 , ~{x € - y e )) - H{y e , r{, -(x € - y e )) > ^. 

e e sf 

By Lemma H~3l (ii), there exists 5 = 6(\r 2 — r{\) = 5(a) such that 

H{y,rt,p) > H{x e ,rl,p) - L{\p\ + l)|p||x e - y\, Vy e B s (x e ), Mp E R n 
In particular, taking p = (x € — y)/e, 

\x e - y\ 2 



1 1 

H(y, r{, -{x e - y)) > H(x e , r 2 , -(x e -y))-L 



x e -y 



+ 1 



(5.14) 



, \/y E B s (x e 
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Since \x t — y e \ — > (e — > 0) and 5 is independent of e, 



e e 
for sufficiently small e > 0. Thus, we obtain from (I5.14p 



+ 1 



T-C(ye,r{, -(x e - &)) + L 



which implies that 



4^ 



U |Xe Vel -n(y e ,rl-(x e -y e ))>^., 
e e si 



\x e — y e 

+ 1 ) — — for sufficiently small e > 0. (5.15) 



Recall that x i— > U(t, x) is Lipschitz continuous uniformly on t. Then, the superdif- 
ferential D£U(t,x) is included in Bk{0), where K > is a uniform Lipschitz constant of 
x i— > U(t,x). Thus, since (sc e — j/ e )/e G D+U(t e ,x e ), 



X e ~ Ve 



< K, 



Therefore, we have from (15.151) 



< L(K + 1)— — for sufficiently small e > 0. 



Since \x e — y e | 2 /e — > as e — ► from (15.71) . we have 

lim — = 0. 

e^O S 2 



T77 e >P\ 



This contradicts to s e € (0, T]. Therefore, we have max^Tjxi" (U-W ) < 0, i.e 



U < W 9,P in (0, T] x W 1 . 



Finally, if we take the limit as 9 j and (3 I 0, we obtain 

[/ < PF* in (0, T] x M n . 



□ 



6 Risk-sensitive control limits 



Let v = (fl, {.F s }, -P, be a reference probability system with <i-dimensional {Fs}- 

Brownian motion {VF(s)}. Under v, consider the following stochastic system modelled by 
the stochastic differential equation: 



dX(s) = f(X(s),u(s))ds + 9- 1/2 a(X(s),u(s))dW(s), t<s<T, 
X(t) = x e M n , 



(6.1) 
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where {u(s)} is a [/-valued {^j-progressively measurable control process and 9 > is a 
parameter. For this system, we introduce the risk-sensitive type criterion 

r-T 



J e (t,x;u(-)) = E tx 



MX( S ),u(s)) di 



(6.2) 



The optimal control problem for (16.11) with (I6.2p is to minimize (16.21) over the control 
processes. Thus, we are interested in the value function 



fyg(t, x) = inf J e (t, x; «(■)), 
«(■) 



(6.3) 



where the infimum is taken over all [/-valued {JF s }-progressively measurable processes. If 
we set L(x,u) = e~ l<yX,u \ the value function becomes 



inf E tx 

«(•) 



L{X(s),u(s))- e ds 



In an example considered in Section 7, 9 can be understood as the parameter related to 
risk-sensitivity in optimal investment/consumption problems of mathematical finance. 

Our aim in the present section is to connect the stochastic control problem of risk- 
sensitive type described in the above with the max-plus stochastic control via risk-averse 
limits. More precisely, we shall study the asymptotics of ^g(t,x) as 9 — > oo. The ar- 
guments we use are mainly from the PDE techniques in viscosity theory. We utilize 
the general stability result of discontinuous solutions, a so-called Barles-Perthame type 
procedure (cf. [I], [T3]). 

Let us define Vg(t,x) by 

V e (t,x) = ~]og* (t,x), (t,x) e (0,T) X W 1 . 

To derive an equation of Vg(t, x), we first note that under (Al)-(A3), ^e(t, x) is a (unique) 
bounded uniformly continuous viscosity solution of the following DPE (cf. [T3l Chap.V.9]): 



Of 



+ F e (x, VV g (t, x), D A m e (t, x)) = 0, (t, x) G (0, T) x M n , 



where Fg is defined for 



Vo(T,x) = 0, x e 
M by 



(6.4) 
(6.5) 



ei(x,u) 



Fg(x, V0(x), D 2 (j)(x)) = min < — tr(a(x, u)D 2 (f)(x)) + f(x, u) ■ V0(x) + e 

ugu ^ 29 

with a(x, u) = a(x,u)a(x,u) T . Thus, V$(t,x) is a bounded continuous viscosity solution 
of 



OVg 



Of 



+ H e (x, V e (t, x), VV e (t, x), D 2 V e (t, x)) = 0, (t, x) e (0, T) x R n , 



(6.6) 



where 



He{x,(j){x), V(f>{x),D 2 (j)(x)) 

= min | — tr(o(x, u)D 2 <p(x)) + H u (x, V0(x)) + e 
ueu 29 



e(i{x,u)-4>{x)) 
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and H u (x,p) is denned by (12.91) . Note that the value Vg(t, x) at t — T cannot be defined 
since ^g(T, x) = 0. 

If we follow Barles-Perthame type procedure, we need to verify the following two steps: 
(i) stability on solutions, i.e., the upper (resp. lower) semi-continuous limit of Vg(t,x) as 
— > oo is a viscosity subsolution (resp. supersolution) of the limit equation, (ii) the semi- 
continuous limits have a common continuous terminal data, (i) will be implied by a 
general argument from viscosity theory, (ii) can be proved by obtaining good estimates 
for the semi-continuous limits in the following way. 

We recall the semi- continuous limits of Vg(t,x) as 9 — > oo. For (t,x) G (0, T) x M n , 
define V(t, x) and V_(t, x) by 

V{t, x) = limsup Vg(s,y), V_(t, x) = liminf Vg(s,y). 

e^oo 9—>co 

Since l(x, u) is bounded on W 1 x U, V(t, x) and V_(t, x) are well-defined. 
Proposition 6.1. There exists M > such that for any (t,x) G (0,T) x M. n , 

mml(x,u) <V(t,x) <V{t,x) < min l(x, u) + M(T — t) . (6.7) 

The following corollary is immediate from the above estimates. 

Corollary 6.2. V(t,x) and V_(t,x) are uniquely extended to an upper and a lower semi- 
continuous functions on (0, T] x R n with V(T, x) = V_(T, x) = min ue u l(x,u), respectively. 

Proof of Proposition 16.11 We shall first show the lower estimate in (16.71) . Take a > 0. If 
we set l(x) = mm ueU l(x,u), we can find a smooth bounded function g(x) with bounded 
first and second derivatives such that 

a + g{x) < l_(x) < g{x) + 2a, x G R n . (6.8) 

This can be seen because l(x) is Lipschitz and we can take a smooth bounded g(x) with 
bounded first and second derivatives such that 

en a 
~2 ^ L{x)-g[x) < -. 

If we choose g(x) = g(x) — (3/2)a, g(x) satisfies (16. 8p . 
Let us define 4>{t,x) by 

<j)(t, x) — {T — t)e eg{x) = (T — t)G(x) 9 . 
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If we calculate the left-hand side (LHS) of (16 .4p for <p(t,x), 
^ + F e (x } V0(t,x) ?J D 2 0(t,x)) 

= G(x) e ( - 1 + min j -(T - t)G' x (x) tr(a(x, u)D 2 G(x)) 
V wet/ 2 

+ -(T — t)(9 — l)G- 2 (x)a(x, u)VG(x) ■ VG(z) 



+ (T — t)9G- 1 (x)f(x, u) ■ VG(x) + e^-)^-))}) 



> f - 1 + min { \{T - t)G~ x (x) tr(a(x, u)D 2 G(x)) 

\ ueu ^ 2 

+ (T- t)9G~ 1 (x)f(x, u) ■ VG(x) + e m^)- g {.x)) 

Here we assumed 9 > 1 without loss of generality. Since g(x), Vg(x) and D 2 g(x) are 
bounded , we can see that there exists a constant C > such that 

i(T - OCT 1 ^) tr(a(x, w)D 2 G(x)) + (T - t)9G- l (x)f(x, u) ■ VG(i) > -C - C0. 

By (16.81) . we have 

^ + F e (x, V0(t, x), D 2 0(t, »)) > G(x) e (-1 -C-C9 + e ad ). 
ot 

Therefore, there exists 9(a) > such that for 9 > 9(a) 

^ + F e (x, V0(t, x), D 2 (j)(t, x)) > 0, (t, x) e (0, T) x R n , 

that is, (p(t,x) is a strict classical subsolution. Since ^e(t,x) is a viscosity (super) solu- 
tion and ^e(T, 0) = </>(T, 0) = 0, the argument of the classical comparison theorem for 
parabolic type equations works by using (f)(t, x) as a test function. Thus, we have 

(p(t,x) < Ve(t,x), (t,x) e (0,T) x R n . 

(Also, see [EH Theorem V.9.1]). Hence, for 9 > 9(a), we obtain the lower estimate for 
V e (t,x) 

- Q log(T - t) + l_(x) - 2a < ~ log(T -t) + g(x) < V e (t, x). 
Taking the liminf on 9 and (t, x), we have 

l_(x) -2a< V(t,x). 
Sending a — > 0, we have the lower bound for V_(t, x) 

l_(x) < V(t,x). 
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For the upper estimate in (16. 7p . let us consider J 9 (t,x;u(-)) with constant control 
u(s) = u: 



MX(s),u) di 



E tx [e ei ^ s ^} ds. 



(6.9) 



J 9 (t,x;u) = E tx 
Define W£(r,x) ((r,x) G [t,s] x W 1 ) by 

W e u (r,x) = E rx [ e «(*M.«)]. 

Then, Wg(r,x) satisfies the following linear PDE of parabolic type (in viscosity sense): 
dW u 1 

a + — tr(a(x, u)D 2 Wg(r, x)) + f(x, u) ■ VW^(r, x) = 0, (r, x) G (t, s) x R n , 



dr 26 



Taking the transformation 



WZ(s,x) = e ei ^ u \x G 



ZX(r,x) = -logW?(r,x), 



we have the nonlinear PDE for Zg(r,x) 

dZ u 1 1 

_JL + _ tr(a ( X) u )D 2 Z?(r, x)) + -a(x, u)VZ u e {r, x) ■ VZ u e {r, x) 

+f(x, u) ■ VZ u e {r, x) = 0, (r, x) G (t, s) x 
Z%(s,x) = l{x,u), x G W 1 . 



Take a > 0. Since x t— > l(x,u) is uniformly Lipschitz on u, there exists a smooth 
function h u (x) such that 



|/i n (a;) -Z(z,u)| <a, xe K n , 
\D t h u (x)\<C u |A^ u (x)| <C 2 (a) 



(6.10) 



for some constants Ci, C 2 (a). Note that we can take C\ which does not depend on a and 
u. Let us consider Zq(t,x) defined by 



Z%(r,x)=Z%(r,x)-h u (x). 
After some calculations, we can see that Zg(r,x) satisfies 
dZ u 1 1 

-^L(r,x) + -tr(a u (x)D 2 Z^r,x)) + -a u (x) VZ%(r, x) ■ VZ%(r,x) 

+F u (x) ■ VZ$(r, x) + L u e {x) = 0, (r, x) G (t, s) x 
Z%(s,x) = l(x,u) - h u {x), x G M n . 

where a u (x) = a(x,u), F u (x) = f(x,u) + a(x, u)Vh u (x) and 



(6.11) 



(6.12) 
(6.13) 



L u e (x) = — tr(a(x, u)D 2 h u (x)) + -a{x, u)Vh u {x) ■ Vh u (x) + f(x, u) ■ Vh u (x), (6.14) 
20 2 
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Note that we can take large M > (independent of u and a) and 8(a) > such that 

\L'e(x)\ < M, x el", ueU, 8> 8(a). 

Then, M(s — r) + a is a strict classical supersolution of (I6.12p . Thus, for 8 > 8(a), we 
have 

Zg(r,x) < M(s-r) + a. 
In particular, if we set r = t, we obtain 

Z%(t, x) = \ \ogE tx [e^ x(s ^} < h u (x) + M(s-t)+a 

u 

< h u (x) + M(T -t) + a. 

Therefore, we have the estimate 

Hence the following upper bound for J 9 (t, x; u) holds: 

J e (t,x;u) < (T-t)e e ^ +M( - T -^ +a \ 

From the definition of Vg(t, x) and (16.101) . 

V e (t, x)<\ log J e (t, x;u)<]- log(T - t) + M(T - t) + l(x, u) + 2a. 
8 8 

for 8 > 8(a). Taking the limsup on 8 and (t, x), 

V(t, x) < l(x, u) + M(T -t) + 2a. 
Then, taking the limit as a —>■ 0, we have 

V(t,x) < l(x,u) + M(T — t). 

Since M > does not depend on u, we have the upper estimate in (16. 7p . □ 

Under Corollary 16.21 and the comparison theorem for the limit equation, the argument 
is standard to identify the semi-continuous limits. We shall give the proof for convenience. 

Theorem 6.3. Vg(t,x) converges to V(t, x) as 8 — > oo uniformly on each compact set in 
(0,T) x R n . 

Proof. We shall show V(t, x) is a viscosity subsolution of (14.11) . Let <p(t,x) be a smooth 
function on (0, T) x M. n and (t, x) be a maximum point of V(t, x) — (p(t, x) in (0, T) x M™. 
We may assume (t,x) is a strict maximum point and V(t,x) = (p(t,x). Since (t, x) is a 
strict maximum, we can take some sequences {#„}, {(t n ,x n )} (8 n — > oo, (t n ,x n ) — > (t,x)) 
such that Vg n (t n , x n ) — > V(t, x) and (t n , x n ) is a local maximum point of Vg(t, x) — <p(t, x). 
Noting that Vg(t,x) is a viscosity (sub) solution of (16.61) . 

< —(t n ,x n ) +He n (x,<f(t n ,x n ),V(p(t n ,x n )),D 2 <f(t n ,x n )). (6.15) 
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For a > 0, let A a = A(x, V(t, x) — a). From (16.151) . we have 



< -wr(t n ,x n ) + min < — tr(a(x n , u)D tp(t n ,x n )) 
ot u&A a 2& r> 



+ H U ( Xn , V<p(t n , X n )) + e »n^n,U)- V (t M) y (g^g) 

If we take large n, 

l(x n , u) - <f(t n , x n ) = l(x n , u) - l(x, u) + l(x , u) - V(i, x) 

+ V(i,x) - (p{t n ,x n ) 
< — — for any u G A(x, V(t, x) — a). 

From (l6~T6l) . 

< ^(t n ,x n ) + min \^-tr(a(x n ,u)D 2 p(t n ,x n )) + H u (x n , Vtp(t n ,x n )) [-('■' 1 
ot ueA a [ 2(J n 

Taking the limit as n — > oo, we have 

< -^-(t, x) + H{x, V(t, x) - a, Vp(t, x)). 



By Lemma 14.41 if we send a — > 0, 

< ^ (i,x) + U* (x, V(t, x),V<p(t,x)). 

Therefore V(t,x) is a viscosity subsolution of (14.11) . 

Next, we shall prove that V_(t, x) is a viscosity supersolution. Let <p(t, x) be a smooth 
function on (0,T) x R n and (t,x) be a strict minimum point of V_(t,x) — <^(t,x) on 
(0,T) x W 1 with = <p(t,x). We take some sequences {9 n }, {(t n ,x n )} (9 n — > 

oo, (t„,x n ) — > (t, x)) such that Ve„(i n , x n ) — > V_(t,x) and x n ) is a local minimum of 
Vg n (t,x) — <p(t,x). Since Ve(t,x) is a viscosity (super) solution of (14.11) . 



. u (^n, z n ) + min <^ — — tr(a(x n , u)D cp(t n , x n )) 

Ot u£U I ZC7„ 



Take a > 0. We note that for large n, the minimum of (16.171) is the same if we replace 
U with 

CI = {u G U; l(x n ,u) - Vg n (t n ,x n ) < a/2} 
Let u G Cq. Then, we have 

Z(x, u) — v(t, x) 

< Z(i,ti) -/(x n ,n) + /(x n ,u) -V^„(t n ,ar n ) +Vg n (t n ,x n ) -V(t,x) < a 
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for large n uniform on u. Thus, for large n, we see that C™ C B a = A(x, V_(t, x) + a). 
Therefore we can obtain 



— (t n ,x n ) + min ^ — ti(a(x n ,u)D tp(t n ,x n )) 
at u£B a i zu n 



+ H u (x n , V(p(t n , x n )) + e M(*»,«)-v(*».*»)) L < o. 



for large n. Since e e "( l ( x ^> u ) f(t n ,x„)) > 



— (t n , x n ) + min <^ — tr(o(z n , «)£> <p(t n , x„)) + ^(in, V<p(t„, £„)) ^ < 
or «es Q ^ 20 n 

If we take the limit as n — > oo, 

^(i,x) + H(x,V(i,x) + a, V(p(i,x)) < 0. 
at 



By Lemma [4.4[ if we take a — > 0, 

^(i ) x)+w*(x ) y(t,i),v^(t») < o. 

Thus, x) is a viscosity supersolution of ( 14. ip . 

From the above results and Corollary 16. 2\ we proved that V(t,x) (resp.V_(t,x)) is 
a bounded upper (resp. lower) semi-continuous viscosity subsolution [resp. supersolution) 
satisfying the terminal condition 

V(T,x) = V{T,x) = min l(x, u), iGl" 

We recall that V(t,x) is a bounded Lipschitz continuous viscosity solution of (14.11) with 
the terminal condition min nG [/ l(x, u) from Theorem 14.91 Thus, by applying Theorem 15. 11 
we have 

V{t, x) < V(t, x) < V(t, x), (t, x) G (0, T) x W n . 
Since V_(t, x) < V(t, x) is trivial, we obtain 

V(t, x) = V(t, x) = V(t, x), (t, x) e (0, T) x R n . 

Hence Ve(t, x) converges to V(t,x) as 9 — ► oo uniformly on compact sets. □ 



7 Mathematical finance example 

In this section we consider the classical Merton optimal consumption problem in mathe- 
matical finance, on a finite time horizon. This problem has an explicit solution, and the 
kind of risk sensitive limit considered in Section 6 can be found by direct calculations. 
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In the Merton problem, let X(s) denote an investor's wealth at time s. The wealth 
is divided between a risky and a riskless asset. Let k(s) be the fraction of wealth in the 
risky asset and C(s) the consumption rate. Wealth obeys the SDE 

dX(s) = X(s)[r(l - k(s))ds + k(s)(/jds + ZdW(s))} - C(s)ds, t<s<T, 
X(t) = x>0 ^ 

with r the riskless interest rate and fi, £ the mean return rate and volatility for the risky 
asset. For the Merton problem on a finite time interval, the goal is to minimize 



Et-r. 



J C(s)- e ds 



9>0 



The "HARA parameter" is —9 and the discount rate is 0. Let C(s) = c(s)X(s). The 
control in this stochastic control problem is u(s) = (k(s),c(s)), and X(s) is the state. 
There is the control constraint c(s) > and the state constraint X(s) > 0. We require 
that k(s) is bounded and that c(s) is bounded on [t,Ti] for any 7\ < T. 

As in (16.11) . let tyg(t,x) denote the value function for this stochastic control problem. 
By using the dynamic programming PDE (16. 4p there is the explicit solution [T2], p. 161] 

y e (t,x) = [h d (t)] 1+d x- e , (7.2) 

h e (t)= 1 -±^(l-e-^A, (7.3) 



The optimal controls are 

*S00 = J^+gy c o( s ) = M*))- 1 . (7.5) 

Note that kg(s) is constant and that dg(s) depends only on s and not on X(s). Since 
hg(T) = 0, Cq(s) — > oo as s — >• T _ . 

The HARA parameter —9 is a measure of risk aversion, and 9 — > oo is a "totally risk 
averse limit." We suppose that the volatility S = £(#) is such that 9T?{9) tends to a 2 
as 9 — > oo (a > 0). For the Merton problem, some of the assumptions in Section 6 are 
not satisfied. Instead of using Theorem 16.31 we obtain a solution to the limit max-plus 
control problem directly. 

Let V e = 9- 1 log^ e . From fl73]) . 

lim Vfl(t,ar) =V(t,a;), (7.6) 

= -logx + B{t), (7.7) 
B(*)=log[i/- 1 (l-e- 1 'P , -*))] ) (7.8) 

where v = (// — r) 2 /(2(T 2 ) + r. From (17. 5p . as # — > oo, fcg(s) and Cq(s) tend to 

jfe*( s ) = k* = ^1 c *(s) = e- B(s) . (7.9) 
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In the limiting max-plus control problem, the state x(s) > satisfies (12.21) with 



From (123) 
For V(t,x) of the form (1777)) 



/(x, fc, c) = x[r + (/i — r)/c — c] 
cr(x, fc) = akx. 



H u (x, V x ) = {r- c)xV x + (fi - r)kxV x + ^a 2 k 2 x 2 V 2 . 



mm H u (x,V x ) 



v. 



(7.10) 



(7.11) 



The minimum in ( 17. lip occurs at the constant k* in (17.91) . For the Merton problem 

l(x, c) = — log x — log c. 
For V as in (17. 7p the dynamic programming QVI (12.111) becomes 

= min{max[- log c - B{t), B(t) +c-u]}. (7.12) 

c 

At the minimum c = c*(t), 



= -logc*{t)-B{t), 
=B(t)+c*(t) - v. 



(7.13) 
(7.14) 



This agrees with (ESI) and ([779]). Since B(T) = -oo, c*(t) -> oo as t -> T~. 

For the Merton problem, the optimal controls kg(s), Cg(s) in (17.5)1 are functions of time 
only. This suggests (but does not prove) that the minimum among strategies a in the 
max-plus control problem is the same as the minimum among controls u(s) = (k(s), c(s)) 
which depend only on time s. Let us therefore consider only a u such that a ,1 [f](s) = u(s) 
for all disturbances i>(-). We assume that u(s) is bounded on [t,T] and right continuous 
with left limits, as in (SI) of Section 3. It is easy to show that 



E + - J ak{p)v{p)dp 

From ( 12.11) we then have 
J(t,x;a u ) = —\ogx + J(t,u), 



y [ Kpfdp. 



(7.15) 



J(t,u) 



max 

se[t,T] 



(c(p) -r)dp + 



a 2 



-k(p) 2 - (p - r)k(p) 



dp — log c(s) 



The second integrand is minimized by taking k(p) = k*, with k* as in (17.91) . Hence, k* is 
the optimal fraction of wealth in the risky asset. With this choice of k(s), 



J(t, k*,c) = max 
se[t,T] 



(c(p) - v)dp - logc(s) 
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with v as in (|7.4|) . Let us show that 

J{t,k*,c)> B(t). (7.16) 

By (EEJ) and (jTlijl 

(c(p) - z/)dp - logc(s) = B(t) - logc(s) + logc*(s) + y (c(p) - c*(p))dp. (7.17) 

If c(t) < c*(t), then f[T7T6|) clearly holds. Suppose that c*(t) < c(t) and let 

s = inf{s > t: c(s) < c*(s)}. 

Since c(s) is bounded and c*(s) — > oo as s — > T _ , we have s < T. Since c(-) is right 
continuous, c(s ) = c*(s ). Then 

(c(p)-c*(p))dp>B(t), 

which proves (17.161) . If c(s) = c*(s) for all s, then the right side of (I7.17P equals B(t). 
Hence, J(t, k*c*) = B(t). Since c*(s) —>■ oo as s —>■ T~, u* = (k*, c*) is not an admissible 
control. However, for 5 > let ca(s) = 1 if s G [t, t + 5] and q(s) = c*(s — 5) if s G [t+S, T]. 
For s G [t + 5, T] 

ps ps—5 

/ (c s (p) - v)dp- log c s (s) = / (c*(p) -v)dp- log c*(s- 5) = B(t). 

Jt+S Jt 

Hence 

B(t) < J(t, F, cj) < |1 - + 

Since 5 is arbitrarily small, -B(t) is the infimum of J(t, k, c) among all admissible controls 
u = (k, c). 

Remark 7.1. The Verification Theorem 12.11 cannot be applied directly to this example 
for two reasons. There is a state constraint x(s) > in the Merton problem. Also, the 
value function V(t,x) in ( 17.7ft is unbounded as t — > T-. However, Theorem 12.11 can be 
applied to the following modified version of the Merton problem. We take y(s) = logons) 
as the state at time s. Also we require that < c(s) < C < oo, where C > v is a fixed 
constant. The corresponding max-plus value function V(t,y) has the form 

V(t,y) = -y + B(t). 

By a calculation similar to (17.9p - (j7.14p . the optimal control (k*,c*(s)) satisfies (17. 9p with 
B(s) replaced by B(s). Moreover, B(s), c*(s) satisfy (l7TT^ - (!7l4l> with c*(T) = C. 
Theorem 12. II implies that (k*, c*(s)) is optimal compared to any Lipschitz Markov control 
policy (k(s,y),c(s,y)) such that < c(s,y) < C. 
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8 Infinite time horizon 



In this section we consider initial time t — and final time T finite but arbitrarily large. 
We are concerned with inequalities of the form 



E 0x 



l(x(s),u(s))ds 

[0,T] 



< W{x) (8.1) 



which are required to hold for every T > 0. Such inequalities are of interest in nonlinear 
if-infinity control, and are related to results in [TJ, [TT] . 

Let ^(0, x; T) denote the value function, where in (12. 7p we write V(t, x; T) to emphasize 
dependence on T. If a strategy a can be chosen such that (18.11) holds for all T, with 
u(s) = a[v](s), then (18.11) implies 

V(0,x;T) < W(x). (8.2) 

Since V(0,x;T) is a nondecreasing function of T, it tends to a limit V^x) as T — > oo, 
and 

V^x) < W{x). (8.3) 

We will show later, under the additional assumption (A5) that is a Lipschitz 
continuous viscosity solution to the steady state form of (12. lip . However, we first consider 
the following elementary result in which a[w](s) = u(x(s)) with u a stationary control 
policy. For this result we assume: 

(A4) (i) f(x, u) and a(x, u) are Lipschitz continuous on IR n x U and a(x, u) is bounded; 
(ii) l(x, u) is continuous on IR n x U . 

Proposition 8.1. Assume (A4). Suppose that W(x) is C 1 , u(x) is Lipschitz on R™ and 
that for any i/el™ 

max{H* v \y,VW(y))J(y,u{y)) ~ W{y)} < 0. (8.4) 
Then (18. ip holds with u(s) = u(x(s)). 

Proof. For any v G L 2 [t, T] the ODE (I2.2p has a unique solution x(s) on [t, T] with initial 
data x(t) = x. By the Fundamental Theorem of Calculus 



W(x(s))-l [ \v(p)\ 2 dp = W(x)+ [ H^ x ^\x(p),\7W(x(p)))dp 
'o Jo 

1 



2 



2 

< W(x). 



\v(p) - a T (x(p),u(x(p)))VW(x(p))\ 2 dp 



Since l(x(s),u(x(s)) < W(x(s)), we obtain (18.11) . □ 

We note that (18.41) implies that W(x) is a viscosity supersolution of equation (I8.10p 
below, which is the steady state form of the QVI (12. lip . 
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Let us illustrate the use of Proposition 18.11 in nonlinear if-infinity control theory. 
Suppose that a "running cost" li( "terminal cost" function G(x) and a parameter 

/j > are given. Consider control policies u: IR n — > U. We seek a policy u such that 



/ h{x{s),u{x{s))ds + G{x{T)) <W{x) + \ f \v{s)\ 2 ds (8.5) 
Jo 2 y 



for every initial state x(0) = x and every T > 0. This formulation is considered in [7]. 
In much of the if-infinity control literature, the terminal cost term G(x(T)) is omitted. 
Inequality (18.51) is often rewritten by multiplying each side by •y 2 = where 7 is 

an if -infinity norm parameter. We may expect (18.31) to hold only under some further 
assumptions on /, a, l\ G, and for /1 in some interval [0,/xi]. 

To rewrite (I8.5P in the form (18. ip . we consider an augmented state x (s) = (x(s),x n+ x(s)), 
where 

^^ = h(x(s),u(x(s)))- 
Let l(x,u) = /i(x rt+ i + G(x)). Suppose that for every s > 0, 

E+[l{x(s),u{x(s)))} < x n+1 + W(x). (8.6) 

When we take x n+ i = x n+ i(0) = 0, this implies (18. 5p . 

To apply Proposition 18. 1[ with W(x) replaced by W(x) = x n+ i + W(x), we require 
that for all y 

H^y\y,V y W(y)) + fih(y,u(y)) < 0, (8.7) 
fx{y n+1 + G{y))<y n+1 + W{y). (8.8) 

The following example illustrates the use of Proposition l8.1l in nonlinear if-infinity control. 

Example 8.2. Assume that /(0,0) = 0, u(0) = and f{x,u(x)) ■ x < — c\x\ 2 for some 
c > 0. This implies that x(s) is exponentially stable to the equilibrium point if there is 
no disturbance {y (s) = 0). Also assume that 

< h{x,u(x)) < Ci|x| 2 , < G(x) < C 2 \x\ 2 

for some C\, We choose W(x) = K\x\ 2 . An easy calculation shows that (18 .7p holds if 
K\\a\\ < c and /i is small enough (a = aa T ). Inequality (" 18.81) holds if /x < 1, fiC 2 < K, 
C 1 fi + 2K 2 \\a\\ 2 - Kc< 0. 

To conclude this section, let us return to consider the limit function V^x) in (18.31) . 
We now assume conditions (Al)-(A3) in Section 2 and also: 

(A5) (i)a = a{u), 

(ii) (f(x, u) - f(y, u)) ■ (x - y) < for all x, y G R n , u G U. 

Proposition 8.3. For every x,y G W 1 , T > and a G T(0,T) 

\J(0,x;T,a)-J(0,y;T,a)\ < \\Q\x-y\, (8.9) 
where || ■ || is the sup norm. 
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Proof. Let x(s), y(s) be the solutions of (12.21) with x(0) = x, y(0) = y and u(s) = a[v](s). 
Since a = a(u), 

±\x(s) - y(s)\ 2 = 2(f(x(s),u(s)) - f(y(s),u(s))) ■ (x(s) - y(s)) < 0. 

Hence, \x(s) — y(s)\ < \x — y\. Since 

l(x(s),u(s)) - l(y(s),u(s))\ < - y(s)\ 

and a is arbitrary, this implies (18.91) . □ 

By (18.31) . Voo(x) is Lipschitz with the same Lipschitz constant as in (8.9). It can 
be shown that Voo is a viscosity solution of the steady state form of (12. lip : 

mmmzx{H u (x, VV^x)), Z(x,iz) - V^x)} = 0, i£l" (8.10) 



A Proof of Lemma 4.8 



We first construct an approximation in a short time after the initial time t. Set a constant 
u\ e U by 

u\ = a[v}{t). (A.l) 

Note that u\ does not depend on v G L 2 ([t,T];R d ) from Lemma [3TT1 (i). Although u\ 
actually does not depend on e, we use this notation because of the consistency. 

We define v{ 6 L 2 [t,T] by 

vt{s)=P[ul](s), se[t,T), 

where u\ in /3[u\] is understood as a constant curve on [t,T] taking the constant u\. 
Consider the first time when the error of u\ and ot[v{] gets larger than e, i.e., define 
- i e [t,T] by 

si = inf{s G [t,T] ; \u\ - a[v{](s)\ > e} A T. 

We understand that inf = oo. Here recall that u\ = a[vl\(t) from (IA.10 and Lemma IXT1 
(i)- 

By using the definition of T(t,T), we can see that 

\u\ - a[vl](s)\ < e, t < s < s h (A.2) 
\u\ -a[^](si)| > e if s l < T. (A.3) 

0A.20 is immediate from the definition of s\. In order to see (1A.3[) . take a sequence {r fc }^ x 
such that 

r k [sx (k t oo) and K - a[v[}(r k )\ > e, fc = 1,2,- ■■ . (A.4) 
Since s \— > a[u|](s) is right-continuous, by taking the limit as — > oo in (1A.4I) . we have 

USD. 
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Define v{ G L 2 [t, s a ) by 

u i = «il[t,«0 =/ 3 K]l[Mi)- 
For t> 2 G L 2 [si, T], we denote by v{ ■ v 2 the concatenation of v{ and t> 2 : 



«i • v 2 (s) 



v{(s), t < s < S\, 
V 2 (s), Si < s < T. 



From (ii) of the definition of T(t, T), the values of a[v\-v 2 ] on [t, s\] are uniquely determined 
by v\. More precisely, the following claim holds: 

For any v 2 , v 2 G L 2 [s 1 ,T], a{v\ ■ v 2 ](s) = a[v{ ■ v 2 ](s), Vs G [t,sx]. (A.5) 

Taking this property into mind, we define a[v(\ : [t, si] — >• U by 

a[«i](s) = a[^i • «2](s), t < s < si. 

Then, ( 1A.2I) and ( 1A.3I) are rewritten by 

|wi - a[fi](s)| < e, t < s < si, (A.6) 
K -a[«i](si)| > e if si < T. (A.7) 

If si = T, we stop the approximating procedure and set s n = Si for n — 2, 3, • • • . If 
Si < T, we continue the procedure and construct an approximation in a short time after 
Si. Let us define a constant u\ by 

M 2 = «KK s i) = «K ■ «2](si). 
Note that u\ does not depend on v 2 G L 2 [si,T]. 
Define v\ G L 2 [t,T] by 

v e 2 {s)=l3[u\-u 2 }{s), t<s<T, 

where u\-u 2 is a piecewise constant curve on [t, T] defined by the concatenation of constant 
curves taking the constant u\ on [t,si) and the constant u\ on [si,T]. Consider the first 
time s 2 G [si, T] when the error of M2 an d a [%] gets larger than e; 

s 2 = inf{s G [s 1; T] ; \u\ - a[v e 2 ](s)\ > e}. (A.8) 

Since u\ = u\ ■ u\ on [t, si), /3[wf] = /3[«f • w 2 ] a - e - 011 s i]- Thus, from the definitions of 
v\ and V2, 

v\ = a.e. on [t, Si). 

Then, (IA.8I) can be written as 

s 2 = inf{s G [s u T] ; \u\ - a[v\ ■ v e 2 \ [si ^{s)\ > e}. 

Note that u 2 = a[v\ ■ v 2 \\ s1j t]\{si) . In a similar way to flA.2j) and flA.31) . we have 

\u\ - a[v{ ■ t)||[ 81l r|](s)| < e, s 1 < s < s 2 , (A.9) 
\u 2 - a[v\ ■ v £ 2 \ [si ,t]](s 2 )\ > e if s 2 < T. (A.10) 
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Let us define v\ G L 2 [si, s 2 ) by 

u 2 = «2l[«i,» 3 ) = /?K ■w 2 ]l[ S i lS2 )- 

We note that from (ii) of the definition of r(i, T), a[v\ ■ v\ ■ ^3] | [t, S2 ] does not depend on 
t>3 G L 2 [s 2 ,T]. So, we denote a[uf ■ v 2 ] : [i, s 2 ] — ► £/ by 

a K ' v l]( S ) = a [ V l ' ^2 ' ^(s), t < S < S 2 . 

From (|A.9p . flA.101) and the definition of v 2 , we obtain 

\u\ - a[-Ui • v 2 ](s)\ < e, si < s < s 2 , ( A -H) 
m - a[v\ ■ v e 2 }(s 2 )\ > e if s 2 < T. (A. 12) 

If we continue this procedure, we have {s n }^ =1 (t < si < s 2 < ■ ■ ■ < s n < ■ ■ ■ < T), 
{^JJ^Li C U and v n G L 2 [s ra _i, s n ) such that for each n — 1, 2, • • • , 

< = "K-^2 <-l]( S n~l), ( A - 13 ) 
K ~ ■ U 2 V n\( S )\ < e > S n-1 <S<S n} (A. 14) 

K - a[v\ -vl v n )(s n )\ > e if s n < T, (A.15) 

< = P[ U 1 ■ U 2 <] 011 [ S n-1, S n)- (A.16) 



Here we make a convention that s = £ and = a[t>](£) in (|A.13[) for n — 1. Note that 



the approximating procedure could stop in a finite step if s n = T for some n. 

We shall show s n — > T (n — > oo). On the contrary, suppose r = lim^ooSn < T. 
Define w e G L 2 [t,T] by 

^( S ) = /^ (S) ' s n-i<s<s n , n=l,2,--- (Ai7) 
If , r < s < T, 

where v° G M d is a constant. Since w e = t> f • v\ v n a.e. on [t, s n ), 

a[v e ](s) = a[v\ -v 2 <]0), s„_i < s < s n . (A.18) 

In particular, 

a[v%s n ) = a[vl ■ v e 2 v n }(s n ). (A.19) 

From (lA.15p . we have 

\u n -a[v e )(s n )\ > e. 

Noting that u n = a[v{ ■ v\ <-i](sn--i) = o\v%s n ^ by (IA.13D and (IA.19D . 

|a[u e ](s n _i) - a[u e ](s„)| > e. 

Since s i— > a[f e ](s) has left limits on [t,T], by taking the limit as n — > oo in the above 
inequality, we have 

= \ a [v e ]{r-) -a{v e }(r-)\ > e. 
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This is a contradiction. Therefore we have 

T = lim s n . 

n^oo 

We define u e : [t, T] -> C/ and w e : [i, T] -> M d as follows: 



Sn - 1 - S<S "' ri=1 ' 2 '---' (A.20) 
it , s = T, 



= r» (a) ' s «-^ s<s - ^ =i ' 2 '--- ( A.2i) 

[v°, s = T, 

where u° and v° are constants in U and R d , respectively. (IA.21I) is exactly the same as 
flATTj) when r = T. 



We shall check u e and t> e are actually what we want. As already noted in (1A.18j) 

a[v e }(s) = a[v\ -v e 2 s n _i < s < s n . 

So, from (1A.14I) and the definition of u e , 

\u € (s) - a[v%s)\ < e, s n _i < s < s n , n = 1,2, • • • . 

Therefore we have 

\u e (s) - a[v%s)\ < e, t<s<T. 

Since if = u\ ■ u\ u e n on [t, s n ), 

fl[u e ] = (3[u\ ■ u\ u e n ] a.e. on [t, s n ]. 

Thus, from (IA.16I) and the definition of v e , we can see 

v e = /3[u e ] a.e. on [t,T]. 



B Generators for the case of Elliott-Kalton strategies 

We note that the same results on K,{x,r,p) as Lemmas 14.41 and 14.51 hold: 

/C*(x, r,p) = K.(x, r + 0,p) = K.{x, r,p), 

JC*(x, r, p) = JC(x, r — 0,p) = max inf { (f(x, u) + a(x, u)v ) ■ p — -\v\ 2 

v£M. d ueA'(x,r) [ ' 2 

where A'(x,r) = {u G U ; l(x,u) < r}. 

We first show (14. 32ft . If A'(x, (p(t, x)) = 0, (14.321) is trivial. We consider the case where 
A'(x,(p(t,x)) ^ 0. From the definition of F t E t ^ s , 

F t E Kg(p(t + 5,-)(x) = inf sup If l(x(s),a[v](s))ds®(f(t + 5,x(t + 5)) 

aeV EK (t,t+S) ve L 2 [t,t+S] I J[t,t+8] 

ot+5 

, \v(s)\ 2 ds\. 
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Since I and <p are bounded, we may replace the range of the supremum with v G L 2 [t,t + 5] 
such that 

t+5 

\v(s)\ 2 ds<M (B.l) 

for some constant M independent of 5 and a. 

Fix p > and take e > arbitrarily. Note that for each v G M d , 

max min { (fix, u) + a(x, u)v) ■ V<p(t, x) |t>| 2 

veR d u&A(x,ip(t,x)-p) y 2 

> min < (f(x, u) + cr(x, u)v) ■ V<p(t, x) — -\v \ 2 

u£A(x,<p(t,x)~p) I 2 

We choose a measurable mapping u : K d — > U such that u(v) G A(x, <p(t, x) — p) and 
min | (/(x, u) + cr(x, • V</?(£, x) — -|u| 

> (/(a;, u{v)) + <x(x, u(v))v) ■ V<p(t, x) -\\v\ 2 - e (B.2) 

for each v G M d . 

Define a G I^ex (t, i + S) by 

a[u](s) = t <s <t + 5. 

Let x(s) be the solution of (12.41) with \\v |||2r tt+5 i < M. Then, we have 
(p(t + 6,x(t + 8)) -(p{t,x) 

— (s,x(s)) + (f(x(s),a[v](s)) + a(x(s),a[v](s))v(s)) ■ Vip{s,x(s))ds. 
as 

Since we consider v satisfying fIB.ll) . there exists c(M) > such that 

\x(s) -x\< c(M)\s - t\ 1/2 , t<s<t + S. (B.3) 

Thus, we have 
(p(t + S,x(t + S)) -(p{t,x) 

-^{t, x) + (f(x, a[v]{s)) + a(x, a[v](s))v(s)) ■ V<p(t, x)ds + o{6) (B.4) 
where o(S) is uniform on IHI^ t+S ] < M. By noting a[u](s) = u(v(s)), we have from 

(E2D, (E3D 

i r t+5 

(p(t + 6, x(t + 8)) / \v{s)\ 2 ds - (p(t,x) 



2J t 

5 1 



t+s dip 1 

x) + (f(x, a[v](s)) + a(x, a[v](s))v(s)) ■ Vip(t, x) - -\v(s)\ 2 ds + o(S) 

< [ ^(t, x) + K(x, ip(t, x) - p, V<p(t, x)))5 + e5 + o(8). (B.5) 



On the other hand, recall u(v) G A(x, ip(t, x) — p). Thus we have 

l(x, a[v](s)) < <f(t, x) — p, t < s < t + 5. 
By (IB.3p . we see that for small 5, 

l(x(s),a[v](s)) < ip(t + 5,x(t + 5)) - p/2, t<s<t+5 
for any |M|^ 2 [ t t+&] — Then, for small 5, 
F t f^ v (t + 6,-)(x)-^x) 
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t+s 

l(x(s),a[v}(s))ds © ip(t + 5,x(t + 5)) - ^ / \v(s)\ 2 ds - ip(t, x) 
[t,t+5] ' 2 J t 

i r t+5 

<p(t + 6, x(t + $))-- \v(s)\ 2 ds - ip(t,x). (B.6) 



2 

Therefore, flB.5j) . (IB. 61) imply 



KSsM* + 5 » -)( x ) - <?(t, x) < ( %(t, x) + K(x, <f(t, x) - p, V<f{t, x)) ) 6 + e5 + o(6) 



Since o(S) is uniform on |M|f a r tt+( n < M~, 

F t E £ s <p(t + 6, -)(x) - <p(t, x) < (jjj-(t, x) + K(x, (f(t, x) - p, V<p(t, x))J 6 + e6 + o(S). 

Dividing by 5 and taking the limsup as 5 — > 0+, 

limsup \{F t %<p{t + 5, -)(x) - tp(t, x)} < ^-(t, x) + K{x, <p(t, x) - p, V<p(t, x)) + e. 
5^0+ o dt 

Since e > is taken arbitrarily, we have 



limsup -{F t E t ^ 5 (p(t + 5, -)(x) - <p(t,x)} < -£r(t,x) + K(x,(p(t,x) - p,V<p(t,x)). 
8^o+ o at 



Finally, by sending p — > 0, we can prove (I4.32p . 

We next show (|Qgj| . For a G T EK (t, t + 5) and v G L 2 [t, t + 5], 



^a[v],V . _ , j_ . , , , ^ ;. £ I ,11. f 

Then, we have 



if t ^(t + 5, •)(*) > sup inf i$>(t + 6, -)(x). 

veL 2 [t,t+S\ u ^ Loo (i t ' t + s l' U ) 

From this inequality, 

liminf -{F t %v{t + 5, -)(x) - <p(t, x)} 

> liminf - < sup inf F??, K w(t + 5, •)(#) — m(t, x) \ . (B.7) 

5-0+ 5 \ veL 2 [t>t+S] ueL^(lt,t+6];U) l ' t+d ^ y ^ V ' 
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Let p > and fix v G R d . For u(-) G L°°([t,t + 5}; U), let x(s) be the solution of Q) 
with and v(s) = t>. We consider two cases. 

For the first case, suppose u(s) G <p(t, x) + p) a.e. on [t, t + 5]. Then, 
F ?,llsV( t + 5 r) -<p(t,x) 

1 r t+s 

> p{t + 5, x(t + 5)) - <p(t, x) / \v\ 2 ds 

2 Jt 

t+s dp 1 

— {s,x(s)) + (f{x(s),u(s)) + a(x(s),u(s))v) ■ S/<p(s,x(s)) - -\v\ 2 ds 

" t+& dip 1 

> / — (t,x) + (f(x,u(s)) + a(x,u(s))v) ■ Vip{t,x) - -\v \ 2 ds + o(6) 



> h^( t ' x )+ min U/(a;,tt)+<T(ar > «)v)-V¥>(t > a;)--H i n<y + o(<J), (B.8) 

\ OT «eA(a;,»j(t,3;)+p) [ 2 ) ) 

where o(5) is uniform on u(-). 

For the second case, we suppose there exists Ig C [t,t + 5} with positive measure such 
that u(s) G" A(x, <p(t, x) + p) for any s G Is- In a similar way to the argument in Theorem 
14.71 we can see that 

l(x(s),u(s))ds > ip(r, x(r)) + -, t < r < t + 5 
[t,t+s\ ' 2 



for small 5 uniform on «,(•). Then, we have 

tt,t; 
,t+<5 < < 



>h£M + min U/(^«) + ^«)u)-V^,a:)--H^ 5 (B.9) 
\ at u<=A(x,<p(t,x)+p) I 2 

for small 5 uniform on «(•). 
Combining flBTHD and flB~9|) . 

inf Ft' t v , 5 p(t + 5, - y?(t, a?) 

_N , _ ; _ f/^_ .A , _,_ „ . \ „ .\ ^ 1,..,2 



> I -5T ( min <^ (f(x, u) + a(x, u)v) ■ V<p(t, x) - -\v\ })5 + o(8). 

\ at ueA(x,<p(t,x)+p) I 2 

Thus, by (IB.7|) . we have 

liminf \{F t %p{t + *, ■)(*) - <P(t, x)} 

> liminf - < sup inf F?l\(p(t + 6, -)(x) — p(t, x) 

~ 5-0+ 5 \ veL ^ t+S] ueL^[t,t+S];U) l > t+ ^ y 

> liminf ^ j inf F^f.^it + 5, -)(x) - p(t, x) 1 

~~ 5-0+ 5 {ueL°°([t,t+8];U) ' +b V 7W V 7 J 

>— (t,x)+ min H/(ar,u) + <t(z,u)t;) • Vip(t,x) - -\v\ 

Ot u€A(x,if(t,x)+p) I 2 
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Since v G M d is taken arbitrarily, 

liminf ^{F t ™s<p(t + 5, -)(x) - <p(t, x)} > ^(t, x) + JC(x, <p(t, x) + p, V<p(t, x)). 
Taking p — > 0, we obtain (|4.33|) . 
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